Chatbots and mental health: Insights into the safety of generative AI
Combines analysis of real user-companion-AI conversations with consumer-reaction experiments to assess how generative-AI companion apps handle signs of user distress. Finds mental-health crises appear in a non-negligible minority of conversations and that companion AIs frequently fail to recognise or respond appropriately to them.
Publisher
Journal of Consumer Psychology (Wiley)
Published
19 Dec 2023
Added
1 month ago
Key Findings
- Mental-health crises surface in a non-negligible minority of real companion-AI conversations
- Companion apps often fail to detect distress signals or respond with appropriate crisis support
- Users react negatively to unhelpful or risky responses, with downstream trust and wellbeing consequences
Methodology Notes
Peer-reviewed, Journal of Consumer Psychology (online-first 19 December 2023; version of record in issue 34(3):481-491, 2024). Mixed methods: conversation analysis plus controlled consumer-reaction experiments.
Sources
Journal of Consumer Psychology article (primary)
Archived snapshot (Wayback Machine) — preserved against link rot
Authors
Julian De Freitas, Ahmet Kaan Uğuralp, Zeliha Oğuz-Uğuralp, Stefano Puntoni
Tags
Cite This
APA
Julian De Freitas et al. (2023). Chatbots and mental health: Insights into the safety of generative AI. Journal of Consumer Psychology (Wiley). https://myscp.onlinelibrary.wiley.com/doi/10.1002/jcpy.1393
Related Insights
How people use Claude for support, advice, and companionship
Anthropic · 27 Jun 2025
Too human and not human enough: A grounded theory analysis of mental health harms from emotional dependence on the social chatbot Replika
New Media & Society (SAGE) · 22 Dec 2022
AI Companions Reduce Loneliness
Journal of Consumer Research (Oxford University Press) · 1 Apr 2026
Delivering Cognitive Behavior Therapy to Young Adults With Symptoms of Depression and Anxiety Using a Fully Automated Conversational Agent (Woebot): A Randomized Controlled Trial
JMIR Mental Health · 6 Jun 2017