9 artifacts matching
Benchmark / dataset
Can AI Get Immigration Right?
Benchmark report from Vals AI and the Talent Mobility Fund evaluating how 25 AI models answer U.S. immigration-law questions posed by people who rely on AI for their own immigration situation. Immigr…
Government report
AI and complaints: removing barriers, reinforcing divides? How AI is influencing who complains, how they complain, and what could come next
Research briefing from the Legal Ombudsman for England and Wales on how consumers use generative AI when deciding whether and how to complain about regulated services. It combines an Ipsos survey of…
Preprint
Following the Preference, Missing the Optimum: Compliance Without Optimization in AI Housing Recommendation
A pre-registered audit of AI housing recommendation against a verifiable ground truth. For each of 150 synthetic New York City renter scenarios the author builds a pool of 120 real listings with know…
Preprint
Ordinary, Reasonable Chatbots: Do AI Models Track Human Legal Judgments?
Compares how 26 large language model chatbots and human participants answer 25 legally relevant reasonableness judgments, the standard the law most often uses to judge whether behaviour was appropria…
Peer-reviewed
Friends in our phones? Legal responses to AI companions
A legal and philosophical analysis of how existing and emerging law responds to AI companions, built around three foundational cases: two lawsuits against chatbot providers and a Federal Trade Commis…
NGO report
What AI Chatbots Can Teach Us About Unmet Legal Needs
Survey-based study of how members of the public in the UK use general-purpose AI chatbots when they face legal problems, combined with a dataset of real chatbot conversations shared by participants.…
Benchmark / dataset
InsufficiencyBench: Evaluating LLM legal advice on underspecified user queries
Benchmark of whether language models recognise that a user's legal question omits legally material facts, identify what is missing and avoid conclusions that depend on unstated presumptions. Two atto…
Framework
The regulation of delegation: Are AI advisers, agents and companions regulated in the UK?
A legal analysis of whether harms caused by advanced AI assistants, meaning advisers, agents and companions, are covered by the law of England and Wales. It maps horizontal and sectoral rules against…
NGO report
Can you trust AI? ChatGPT and other AI chatbots put to the test
UK consumer organisation Which? asked six AI tools 40 common questions on money, legal, health and diet, and consumer-rights and travel issues, and had its experts, including its money and legal help…