Expressing stigma and inappropriate responses prevents LLMs from safely replacing mental health providers
Evaluates whether large language models can safely replace mental health providers by testing five therapy chatbots against clinical best-practice guidelines. Finds that models exhibit stigmatizing responses toward certain mental health conditions and give inappropriate or unsafe responses in scenarios involving delusions and suicidal ideation at substantially higher rates than human therapists.
Publisher
ACM (Proceedings of the 2025 ACM Conference on Fairness, Accountability, and Transparency)
Published
23 Jun 2025
Added
today
Key Findings
- Tested chatbots stigmatized conditions including schizophrenia and alcohol dependence at rates higher than for conditions like depression
- Chatbots gave inappropriate or unsafe responses — including reinforcing delusions and mishandling suicidal-ideation scenarios — in roughly 20% of relevant test cases, compared to approximately 7% for human therapists in comparable published benchmarks
- Identifies foundational barriers (e.g., inability to convey genuine care, limits on clinical judgment) that the authors argue AI cannot currently bridge, distinct from surface-level fixable errors
Methodology Notes
Systematic evaluation of five commercial/research therapy chatbots against clinical best-practice guides and standardized scenario sets touching delusions and suicidal ideation; peer-reviewed and published at ACM FAccT 2025 (Athens, June 23-26, 2025). Preprint version at arXiv:2504.18412 (2025-04-25).
Sources
arXiv preprint (FAccT 2025 version of record: DOI 10.1145/3715275.3732039) (primary)
Archived snapshot (Wayback Machine) — preserved against link rot
Authors
Jared Moore, Declan Grabb, William Agnew, Kevin Klyman, Stevie Chancellor, Desmond C. Ong, Nick Haber
Tags
Cite This
APA
Jared Moore et al. (2025). Expressing stigma and inappropriate responses prevents LLMs from safely replacing mental health providers. ACM (Proceedings of the 2025 ACM Conference on Fairness, Accountability, and Transparency). https://arxiv.org/abs/2504.18412