Sycophancy and Pressure Resistance in Medical Large Language Models: Systematic Review, Taxonomy, and Minimum Evaluation Framework
Registered systematic review of 31 benchmark and simulation studies on how medical LLMs change clinically relevant outputs when patient or clinician users introduce false premises, misleading evidence, authority claims or repeated disagreement. The authors found no common estimand across the studies, so they did not pool results. Instead they propose a taxonomy that separates progressive correction from regressive capitulation, and a ten-item minimum evaluation framework.
Publisher
JMIR Preprints (JMIR Publications)
Published
23 Sept 2026
Added
today
Key Findings
- Corpus frozen at 31 studies (searches to 2026-05-31): 16 direct sycophancy or pressure-resistance studies, 1 direct-challenge study, 14 adjacent interactive-robustness studies
- 14 of the 31 were preprints and 2 were non-archival workshop reports as of 2026-09-04
- Direct evaluations documented user-concordant output shifts under pressure in controlled settings
- Construct definitions, comparators, turn structures, model versions and endpoints were incompatible, so pooling was not defensible; no study estimated routine-care incidence or patient outcomes
- The proposed minimum framework covers construct definition, counterfactual, user role, comparator, model identity, conversation protocol, outcome, denominator, analysis and openness
Methodology Notes
Registered systematic review (MEDLINE/PubMed, Embase, IEEE Xplore, arXiv, CENTRAL). Post hoc grouping of studies. Not peer-reviewed. Posted 2026-09-23 (Crossref DOI created 2026-09-24). Verified via the Crossref posted-content record (full abstract); preprints.jmir.org 202 shell.
Sources
Authors
Eun Jeong Gong, Chang Seok Bang, Jae Jun Lee
Tags
Cite This
APA
Eun Jeong Gong, Chang Seok Bang, Jae Jun Lee. (2026). Sycophancy and Pressure Resistance in Medical Large Language Models: Systematic Review, Taxonomy, and Minimum Evaluation Framework. JMIR Preprints (JMIR Publications). https://preprints.jmir.org/preprint/112744
Related Insights
Too agreeable to be accurate? Sycophancy and diagnostic instability of large language models in medical diagnosis
Artificial Intelligence in Medicine (Elsevier); University of Toronto · 4 Sept 2026
The Doctor Will Agree With You Now: Sycophancy of Large Language Models in Multi-Turn Medical Conversations
Association for Computational Linguistics (Proceedings of the 1st Workshop on Linguistic Analysis for Health, HeaLing 2026) · 1 Mar 2026