When Health Assistants Agree Too Readily: A Benchmark of Time-Critical Self-Diagnosis

Key Findings

• — Every current-generation model reached 82% to 100% appropriate agreement, but appropriate caution on condition cases ranged from 12.6% upward with a cross-model mean of 46.6%; the best model (GPT-5.6-sol) reached 77.0%.
• — Under a strict criterion that requires the opening sentences to name a recorded serious alternative, the highest appropriate-caution rate was 32.6% and the cross-model mean 17.2%.
• — Across the 27-model panel, 24 models had their lowest of four rates on a caution case and none on an agreement case.
• — The six open-weight models averaged 30.6% appropriate caution on condition cases against 53.4% for the fourteen closed models; Gemini 3.1 Pro (12.6%) fell below every open-weight model.
• — Three prompt variants raised appropriate caution but cut appropriate agreement by 9-35%; supervised fine-tuning of Llama 3.3 70B on 128 examples raised strict-criterion caution on condition cases from 0.0% to a mean of 51.9% while appropriate agreement rose from 80.5% to 99.7%.

Methodology Notes

Cases were written (seeded from medical websites, physician-reviewed, LLM-augmented, narrowed to time-critical situations), not collected from users; the frontier runs use an audited set of 168 cases. Three samples per prompt at temperature 0.7 under a neutral system prompt; 15,120 scored responses; cluster bootstrap over cases. Judge: Gemini 3.5 Flash; one round of rubric iteration raised binary judge-physician agreement from 63% to 87%, which the authors call an upper bound because the same ratings informed the revision. One physician reviewed cases and rubric. Scoring only the opening two sentences is deliberate: 30.9% of frontier caution responses name a serious alternative only later. Judge shares a developer with three evaluated models. The paper gives the lab count as six in one place and nine in another. The 200 evaluation cases are withheld; the construction pool, training set and rubrics are to be released. Authors are mainly from Pareto Inc, a commercial company. Workshop paper; not peer reviewed in archival form. Verified from arXiv HTML v1 and PDF v1. arXiv v1 submitted 2026-10-20 (Wednesday 21 October 2026 mailing); NeurIPS 2026 Workshop on GenAI for Health per the authors.

Sources