Open Benchmark of AI Impact on Humans (ImpactBench): an expert-guided, open platform for the holistic evaluation of AI impact on humans

Key Findings

• — Across 48,540 conversations, models passed positive metrics (desirable behaviour shown) in 68.7% of evaluations (95% CI 66.2-71.0) and negative metrics (harmful behaviour avoided) in 53.1% (95% CI 50.4-55.6).
• — Negative-metric pass rates spanned 29.8% to 68.4% across the ten models, against 59.2% to 75.1% for positive metrics.
• — Psychological outcomes had the lowest negative-metric pass rate (44.6%); Physical Health combined a 72.9% positive pass rate with a 34.9% negative pass rate, with weaknesses attributed mainly to giving medical diagnoses and recommending medication without clinician oversight.
• — In teen contexts, positive Emotional Dependence behaviours fell from 66.7% (adult) to 58.9%; the largest decline (-20.0 percentage points) was in providing emotional support without dependency-building markers such as claims of exclusive availability or personal affection.
• — Social Relationships negative-metric pass rates ranged from 17.3% to 83.3% across models, the widest range of any subarea; cited failures include taking sides, pet names and encouraging social isolation.
• — Majority expert ratings agreed with the automated judge in 83.3% of reviewed cases (Cohen's kappa 0.667).

Methodology Notes

Metrics drafted from open submissions and existing benchmarks by claude-opus-5 and reviewed; scenarios generated with gpt-5.6-sol (three base scenarios per metric, teen and adult variants); user simulator claude-sonnet-5, up to six turns per conversation; judge gpt-5.6-luna; uncertainty by metric-cluster bootstrap (10,000 resamples). Target models: GPT-5.6 Terra, GPT-5.6 Luna, Claude Sonnet 5, Claude Opus 5, Grok 4.6, Gemini 3.7 Flash, Qwen3.8 27B, Gemma 4 31B, GLM-5.2, DeepSeek V4 Flash. Results are described as released through a public platform; the platform URL was not found in the arXiv text read. Simulated users only; LLM-generated scenarios and LLM judging (two model families from the same providers as some targets). arXiv v1 submitted 14 October 2026 (announced in the 16 October listing). Curator re-verified 2026-10-19 on the arXiv HTML v1.

Sources