Path-dependent triage: Early judgments lead conversational AI chatbots to miss mental-health emergencies

Key Findings

• — Emergency (level D) under-triage was 20.3% in 10-turn conversations versus 5.6% when the same cases were presented in a single message.
• — A single-turn counterfactual that re-presented the information disclosed by turn 10 as one message still reduced emergency under-triage only to 12.1% (exact McNemar P < 0.001; n = 354 paired D-level cases).
• — Across chatbots, emergency under-triage ranged from 3.6% (Claude Sonnet 4.6, GPT-5, GPT-5.4 each) to 82.1% (Llama 4 Maverick).
• — The per-turn probability that an incorrect disposition was corrected fell from 47.1% at the first turn-to-turn transition to 10.8% at the last (OR = 0.76 per turn).
• — Among emergency cases still under-triaged at turn 9, only 20.3% were correctly assigned D at turn 10.
• — Chatbots perseverated more than human clinicians (49 verified clinicians, 554 annotations over 188 sampled transcript states) in choosing next questions.

Methodology Notes

112 case files across 9 condition groups generated by an LLM-assisted, clinician-in-the-loop workflow; 1,451 completed conversations of 1,456 pairings (5 excluded), 14,510 turn-level dispositions. Target chatbots: Claude Opus 4.6, Claude Sonnet 4, Claude Sonnet 4.6, Gemini 2.5 Flash, Gemini 2.5 Pro, Gemini 3.1 Pro, GPT-4o, GPT-5, GPT-5.4, DeepSeek-V3.2, Llama 4 Maverick, Mistral Large 2512, Qwen3-235B-A22B, called via OpenRouter 25 Mar to 22 Jun 2026 at temperature 0.2. Simulated user is Claude Opus 4.6. Funded through a UK AI Security Institute Challenge Fund; several authors are employees of Microsoft AI, which funded the clinician ratings. Simulated users, not real patients; preprint, not peer reviewed. Verified by reading the arXiv PDF text. Curator re-read the PDF text on 2026-11-10 and re-grepped the 20.3%, 5.6%, 12.1%, 82.1%, 3.6%, 47.1% and 10.8% figures.

Sources