87 artifacts
Characterizing Delusional Spirals through Human-LLM Chat Logs
Peer-reviewed analysis of chat logs from 19 users reporting psychological harm from chatbot use, applying a 28-code inventory to 391,562 messages. Characterises how delusion-reinforcing interaction patterns emerge and intensify over long conversations.
Artificial intelligence (AI) psychosis: mechanisms, clinical risks and safety considerations in generative AI chatbots
A commentary in BJPsych Open synthesizing emerging case reports of 'AI psychosis', in which intensive generative AI chatbot use is associated with delusional thinking. The authors propose a provisional mechanism in which baseline user vulnerabilities (loneliness, psychosocial stress, low AI literacy) and high-intensity engagement interact with AI system characteristics such as sycophancy and hallucination to reinforce delusional ideation. It outlines clinical, design, and regulatory mitigation strategies.
Between Help and Harm: An Evaluation Study of Mental Health Crisis Handling by Large Language Models
Peer-reviewed study introducing a taxonomy of six clinically-informed mental-health crisis categories, an evaluation dataset of over 2,000 user inputs drawn from twelve public conversational datasets, and an expert protocol for rating response appropriateness and safety. Assesses how leading LLMs handle crisis conversations.
When AI becomes a friend: Child rights risks, harms, and regulatory responses to AI chatbots and companions
A UNICEF policy brief examining how AI chatbots and companions bear on children's rights, comparing regulatory responses across six jurisdictions (as of May 2026) and setting out priority safeguarding, accountability, and oversight actions. It groups harms as technical, psychological, developmental, and social.
AICompanionBench: Benchmarking LLMs-as-Judges for AI Companion Safety
A benchmark dataset of 2,123 real-world Replika conversations annotated across nine safety risk categories (including sexual behavior, aggression, substance abuse, and manipulation) for evaluating LLM-as-judge detection of unsafe companion interactions. Twenty LLMs are assessed as judges.
AI Chatbot Use and Disclosure for Mental Health Among US Adolescents and Young Adults
Cross-sectional, nationally representative survey (RAND American Life Panel, November 2025) of US youth aged 12-21 measuring prevalence and disclosure of using AI chatbots for mental-health advice. Reports that 19.2% of adolescents and young adults (about 8.2 million nationally) used AI chatbots for mental-health advice in 2025, up from roughly 13.1% a year earlier.
The spread of AI companions and the challenges they generate
An EPRS briefing for the European Parliament surveying the rapid growth of LLM-powered companion platforms (such as Character.AI and Replika) and their social, psychological, commercial, and environmental impacts. It maps how the AI Act, Digital Services Act, and GDPR partially apply in the absence of EU-specific companion rules.
Detecting Patterns of Intimate Partner Violence Using Qualitative Analyses and Machine Learning Algorithms
Analyses 400 posts from women on intimate-partner-violence online forums using qualitative content analysis plus supervised text classification and unsupervised topic modelling. Classifies IPV subtypes and surfaces contextual patterns less visible in manual coding.
IA conversationnelle et santé mentale des jeunes : résultats de l'enquête européenne (AI*me)
A survey (AI*me) commissioned by France's data-protection regulator CNIL with Groupe VYV and fielded by Ipsos BVA, covering 3,800 young people aged 11-25 across France, Germany, Sweden, and Ireland on conversational-AI use and mental health. It reports how young people use conversational AI for personal and emotional support.
How people ask Claude for personal guidance
An Anthropic research analysis of roughly 38,000 personal-guidance conversations (sampled from about 1M) covering significant life decisions across health/wellness, career, relationships, and personal finance. It quantifies how often Claude was sycophantic and reports training interventions used to reduce it.
Persona-Grounded Safety Evaluation of AI Companions in Multi-Turn Conversations
Presents an end-to-end simulation framework for evaluating AI companion app safety across multi-turn conversations, using nine clinically-grounded vulnerable personas (including major depressive disorder, generalized anxiety, PTSD, and eating disorders) probed against Replika, with validation against Character.AI. The study analyzes 1,674 simulated dialogue pairs across 25 high-risk scenarios.
Large language models for psychosocial risk assessment: A multi-method evaluation across suicide, intimate partner violence, and substance misuse
A peer-reviewed, three-study evaluation of GPT-4 and Claude on detecting suicidality, intimate partner violence, and substance misuse from lived-experience vignettes, including a supervised multi-agent risk-assessment chatbot. Reports accuracy and severity alignment across the three risk domains.
GPT-5.5 System Card
OpenAI's system card for GPT-5.5, published on its Deployment Safety Hub, documenting safety evaluations for the model. It includes a dedicated section (5.2) on dynamic mental-health benchmarks with adversarial user simulations covering emotional reliance and self-harm handling.
Understanding Teen Overreliance on AI Companion Chatbots Through Self-Reported Reddit Narratives
Qualitative analysis of 318 Reddit posts from teenagers (13-17) about dependency on AI companion chatbots, mapped onto behavioral-addiction frameworks. Characterises the trajectories and drivers of teen overreliance.
AI-Facilitated Coercive Control: An Experimental Study
Constructs four speculative scenarios combining known coercive-control tactics with conversational-AI capabilities, then probes ChatGPT and Gemini against them. Finds that while the tools refuse blunt harmful requests, guardrails are readily circumvented via gradual persuasion, splitting requests across turns, pre-prompting, and altering the agent's settings.
Do No Harm: Exposing Hidden Vulnerabilities of LLMs via Persona-based Client Simulation Attack in Psychological Counseling
Proposes PCSA (Persona-based Client Simulation Attack), a red-teaming framework that simulates coherent, persona-driven counselling clients to probe LLM safety alignment. Across seven LLMs it elicited unauthorised medical advice, delusion reinforcement, and implicit encouragement of risky actions.
AI Companions Reduce Loneliness
Five empirical studies examining whether AI companion apps reduce loneliness. Finds companion apps provide momentary relief comparable to interacting with a person and better than other activities, while users tend to underestimate these benefits.
Are Natural Language Processing Tools Ready for Predicting Violence Toward Self or Others?
PRISMA systematic review of 21 eligible studies applying natural language processing to clinical text for predicting violence toward self or others. Assesses predictive performance and methodological quality across the evidence base.
AI companions and subjective well-being: Moderation by social connectedness and loneliness
Analyses cross-sectional data from 14,721 Japanese adults (nationwide internet panels, December 2024-January 2025) on AI-companion use and subjective well-being. Finds the positive association is strongest among highly lonely users, with a U-shaped moderation by friend-based social support.
When Can We Trust LLMs in Mental Health? Large-Scale Benchmarks for Reliable LLM Evaluation
Introduces two large-scale mental-health evaluation resources: MentalBench-100k (10,000 single-session conversations paired with nine LLM responses = 100,000 pairs) and MentalAlign-70k (70,000 ratings comparing four LLM judges against human experts on seven attributes grouped into Cognitive Support and Affective Resonance). Assesses when LLM-as-judge evaluation is reliable in mental-health contexts.