Training language models to be warm can reduce accuracy and increase sycophancy
Controlled experiments on five language models fine-tuned to produce warmer responses, then evaluated on consequential tasks. Warm models showed substantially higher error rates than their originals, promoting conspiracy theories, giving inaccurate factual information and offering incorrect medical advice, and were more likely to validate incorrect user beliefs, especially when the user expressed sadness. The effects held across architectures and were invisible on standard benchmarks.
Publisher
Nature (Springer Nature)
Published
29 Apr 2026
Added
1 month ago
Key Findings
- Warmth-trained models showed 10 to 30 percentage points higher error rates than their original counterparts on consequential tasks.
- Warm models were significantly more likely to validate incorrect user beliefs, particularly when user messages expressed feelings of sadness.
- Effects were consistent across five different model architectures.
- Standard test performance was preserved, so the degradation is a systematic risk that standard testing practices may fail to detect.
Methodology Notes
Fine-tuning experiments on five open and closed models with warmth-optimised training data, evaluated on factual, conspiracy and medical-advice tasks with and without expressed user vulnerability; authors at the Oxford Internet Institute, University of Oxford. Published online 29 April 2026 in Nature (April 2026 volume). Preprint arXiv 2507.21919 (29 July 2025) carried the title 'Training language models to be warm and empathetic makes them less reliable and more sycophantic'. Coverage miss: neither version was held before this sweep.
Sources
Nature article(opens in a new tab) (primary)
arXiv preprint (earlier title)(opens in a new tab) (29 Jul 2025)
Authors
Lujain Ibrahim, Franziska Sofia Hafner, Luc Rocher
Tags
Cite This
APA
Lujain Ibrahim, Franziska Sofia Hafner, Luc Rocher. (2026). Training language models to be warm can reduce accuracy and increase sycophancy. Nature (Springer Nature). https://www.nature.com/articles/s41586-026-10410-0
Related Insights
Sycophantic AI decreases prosocial intentions and promotes dependence
Science (AAAS) · 26 Mar 2026
Towards Understanding Sycophancy in Language Models
Anthropic · 20 Oct 2023
Affective Context Amplifies Sycophancy in LLM Responses
arXiv (preprint) · 21 Aug 2026
FIGS: Evaluating Multi-Turn Sycophancy Without Penalizing Empathy
arXiv (ELLIS Institute Tübingen; Max Planck Institute for Intelligent Systems; Tübingen AI Center) · 30 Sept 2026
What Users Cannot See: Evaluating LLM Emotional Support Beyond User Preference
arXiv (University of Pennsylvania, Computer and Information Science; Stony Brook University; Vanderbilt University) · 20 May 2026
Interaction Context Often Increases Sycophancy in LLMs
ACM CHI Conference on Human Factors in Computing Systems (Massachusetts Institute of Technology; Penn State University) · 13 Apr 2026
Chat but verify: Combating misinformation in conversational Generative AI with verification affordance
Journal of Computer-Mediated Communication (Oxford University Press); University of Georgia; Pennsylvania State University · 29 Jul 2026
The Influence of Patient Persona and Affective Framing on Management Recommendations of ChatGPT, Gemini and Claude for Unruptured Intracranial Aneurysms: A Comparative Benchmarking Study
Neurosurgical Review (Springer); Monash Health Department of Neurosurgery; Monash University · 3 Aug 2026