People Overtrust AI-Generated Medical Advice despite Low Accuracy
Experimental study from MIT Media Lab, Stanford, Cornell Tech/BIDMC OpenNotes and IBM Research on how non-experts perceive medical responses written by doctors on an online health platform versus responses generated by a large language model that physicians had labelled high or low accuracy. 300 participants rated validity, trustworthiness and completeness, and reported whether they would follow the advice. The study also compared expert raters and varied the stated source of the response.
Publisher
NEJM AI (Massachusetts Medical Society); MIT Media Lab
Published
13 May 2025
Added
today
Key Findings
- Participants could not effectively distinguish AI-generated responses from doctors' responses and preferred AI-generated responses, rating high-accuracy AI responses significantly more valid, trustworthy and complete than doctors' responses
- Low-accuracy AI-generated responses on average performed very similarly to doctors' responses on perceived validity, trustworthiness and completeness
- Participants indicated a high tendency to follow potentially harmful low-accuracy AI advice and to seek unnecessary medical attention as a result, a reaction comparable with or stronger than their reaction to doctors' responses
- High-accuracy AI responses were trusted more when labelled as coming from a doctor, and expert raters scored AI responses significantly higher when the source was unknown
Methodology Notes
Online experiment, n = 300 non-expert participants plus expert (physician) evaluations; doctor responses came from an online health care platform; AI responses were labelled high or low accuracy by physicians. Details of the model, question set and sampling are in the paywalled article and were not read; only the structured abstract (PubMed 42932543, indexed October 2026) and the OpenAlex record were available. Published online 13 May 2025 in NEJM AI 2(6), June 2025. Older miss: PubMed indexed the NEJM AI back catalogue between 20 and 27 October 2026. Verified via the PubMed record (efetch, 2026-10-29); ai.nejm.org blocks automated reads.
Sources
NEJM AI article(opens in a new tab) (primary)
PubMed record (structured abstract)(opens in a new tab)
Archived snapshot (Wayback Machine)(opens in a new tab) — preserved against link rot
Authors
Shruthi Shekar, Pat Pataranutaporn, Chethan Sarabu, Guillermo A. Cecchi, Pattie Maes
Tags
Cite This
APA
Shruthi Shekar et al. (2025). People Overtrust AI-Generated Medical Advice despite Low Accuracy. NEJM AI (Massachusetts Medical Society); MIT Media Lab. https://ai.nejm.org/doi/full/10.1056/AIoa2300015