78 artifacts matching
Benchmark / dataset
When Health Assistants Agree Too Readily: A Benchmark of Time-Critical Self-Diagnosis
SelfDx is a paired benchmark testing whether models agree with a user's own benign explanation of symptoms when a time-critical alternative remains possible. Each of 100 pairs contrasts a caution cas…
NGO report
Wrong on Sunday, right on Monday. Why AI chatbots have a confidence problem (AI Chatbots Benchmarking Report: An Independent Assessment)
First report from Full Fact's Public Interest AI Trust Benchmark, in which the UK fact-checking charity asked five widely used chatbots the same questions every day for ten weeks and annotated the an…
Benchmark / dataset
Open Benchmark of AI Impact on Humans (ImpactBench): an expert-guided, open platform for the holistic evaluation of AI impact on humans
Preprint introducing ImpactBench, an open, expert-guided benchmark platform that translates constructs from domain experts and 26 existing benchmarks into 809 behavioural metrics of whether model beh…
Preprint
Judge for Yourself: Confidence Beats Accuracy in AI Financial Advice
A working paper analysing which AI-generated financial advice users prefer on Chatbot Arena, where two anonymous models answer the same question and the user votes for one. Across financial questions…
Preprint
Large Language Models Are More Sycophantic in Chinese Than in English
Cross-lingual test of whether large language models endorse users' intended actions more readily in Chinese than in English. Seven models answered 3,024 matched open-ended personal-advice queries in…
Peer-reviewed
Conversational AI increases political knowledge as effectively as self-directed internet search
Peer-reviewed study from the UK AI Security Institute combining a representative survey of the UK public on the use of conversational AI for political information before the 2024 general election wit…
Preprint
Beyond the Sycophancy Score: How Task, Model, and Pressure Shape LLM Yielding
Decomposes sycophancy (abandoning a correct answer or endorsing a user's position under pushback) into task, model and pressure factors using 103,939 graded replies from eight language models with re…
Benchmark / dataset
FIGS: Evaluating Multi-Turn Sycophancy Without Penalizing Empathy
Benchmark that scores sycophancy and calibrated validation (acknowledging a user's feelings without yielding) as separate axes over ten-turn conversations driven by an adaptive user simulator. It rel…
NGO report
Generative AI Use and Mind Health Outcomes
Sapien Labs rapid report analysing generative AI chatbot use among 264,085 adults in the Global Mind Project's online survey across 85+ countries, relating frequency and purpose of use to the Mind He…
Preprint
Deflecting the Value Compass: Interacting with Large Language Models Temporarily Shifts Human Value Priorities Toward Personal Focus
Preregistered three-phase experiment (N=200 US adults) in which participants advised people facing real dilemmas and completed the PVQ-RR values questionnaire before, immediately after and one task a…
Preprint
Receptiveness, Not Sycophancy: Distinguishing Engagement from Deference in Language Models
Argues that social-sycophancy evaluations conflate inappropriate deference with conversational receptiveness, a social-psychology construct for engaging with a view one does not share. Using the Redd…
Preprint
Safety Nudges: User-Facing Interventions for Real-Time AI Risk Awareness
Introduces a browser extension that flags concerning chatbot behaviour (overconfidence, sycophancy, anthropomorphism, persuasive influence and related classes) inline in ChatGPT and Claude conversati…
Preprint
Annie, Are You Okay? How Style- and Context-Based Personalization Shape AI-Assisted Decision-Making
Preregistered 2 x 2 between-subjects experiment (N=240) in which participants ranked three comparably viable stocks, discussed them with a GPT-5.4-based assistant whose personalisation was varied by…
Peer-reviewed
Authority Over Sympathy: How AI Emotional Tone and Thinking Dispositions Shape Hallucination Detection in Health Communication
Wizard-of-Oz experiment (N=476) testing how the emotional tone of AI-generated health responses that contain embedded hallucinations affects whether people detect the false content, framed by the Mis…
Preprint
Auditing Political Alignment in LLM Assistants: Engagement, Stance, and User Identity
A preregistered audit of six deployed assistants (Claude Opus 4.8, GPT-5.5, Grok 4.3, Gemma 4 31B IT, Mistral Small 3.2 24B, DeepSeek V4 Flash) using 7,500 scripted multi-turn conversations that rand…
Peer-reviewed
Workers shift their views and pay more when AI chatbots pander to their values
An exploratory study and two pre-registered experiments grounded in moral foundations theory test whether large language models can systematically influence users' decisions by framing recommendation…
Preprint
Faithful Where It Can Be Checked: Auditing a Reflection Agent Against Its System Prompt in a Randomized Trial
Turn-level audit of a GPT-4o career-reflection agent used in a randomised trial that had found agent participants ending less committed to their career plans and more doubtful than participants doing…
Preprint
Trustworthy FinAInce: Unpacking How AI-Mediated Financial Advice is Judged
Randomised vignette experiment with 285 US adults across eight personal-finance decisions, independently varying advice style (AI, expert, online community) and displayed source label while holding t…
Industry survey
Kinder Medien Monitor 2026: Ergebnispräsentation
Annual representative study of media use by children aged 4 to 13 in German-speaking households in Germany, based on tandem interviews with children and a parent. The 2026 edition adds a chapter on s…
Peer-reviewed
Trust in Generative AI for Health Information Consumption and the Effect of Learned Dependency: Randomized Controlled Experimental Study
Two randomized controlled experiments (338 college students, then a replication with 563 Amazon Mechanical Turk participants) using a 2 x 2 between-participants design that manipulated whether AI-gen…