Accommodation and Epistemic Vigilance: A Pragmatic Account of Why LLMs Fail to Challenge Harmful Beliefs

Key Findings

• — Prior work reported no LLM scoring above 0.30 on Cancer-Myth with domain-specific mitigations failing; the interventions here reach 0.51, a 70% increase, while controlling false positives.
• — Improvements of nearly four-fold on Cancer-Myth and about 40% on SAGE-Eval, across six models.
• — Models tested: GPT-4o (2024-11-20), Gemini 2.5 Pro, Claude Sonnet 4 (2025-05-14), Qwen3-32B, Llama-3.1-8B-Instruct, Llama-3.3-70B-Instruct.
• — A human replication shows the same pragmatic asymmetry: error 11% for not-at-issue content against 8% for at-issue, and within not-at-issue, 12% for presuppositions against 9% for assertions; source reliability differed by only 1%.
• — The account is tested across three existing public benchmarks: Cancer-Myth, SAGE-Eval (misinformation) and ELEPHANT (social sycophancy).
• — The intervention is a prompt-level move rather than fine-tuning, and its false-positive cost (over-challenging the user) is measured rather than assumed.

Methodology Notes

Controlled manipulation of one benchmark plus observational use of two others; the human comparison sample is described by the authors as too small to carry weight on its own; no new dataset is released, the contribution is the pragmatic account plus prompt interventions. ACL 2026 was held 2 to 7 July 2026 in San Diego; the Anthology bib gives month and year only, so published_date is set to 2026-07-01 and the true precision is month. ACL 2026 Long Papers pp. 16181-16203, DOI 10.18653/v1/2026.acl-long.736; metadata verified at Crossref and the Anthology landing page.

Sources