18 artifacts matching
Artificial intelligence (AI) psychosis: mechanisms, clinical risks and safety considerations in generative AI chatbots
A commentary in BJPsych Open synthesizing emerging case reports of 'AI psychosis', in which intensive generative AI chatbot use is associated with delusional thinking. The authors propose a provisional mechanism in which baseline user vulnerabilities (loneliness, psychosocial stress, low AI literacy) and high-intensity engagement interact with AI system characteristics such as sycophancy and hallucination to reinforce delusional ideation. It outlines clinical, design, and regulatory mitigation strategies.
AI Chatbot Use and Disclosure for Mental Health Among US Adolescents and Young Adults
Cross-sectional, nationally representative survey (RAND American Life Panel, November 2025) of US youth aged 12-21 measuring prevalence and disclosure of using AI chatbots for mental-health advice. Reports that 19.2% of adolescents and young adults (about 8.2 million nationally) used AI chatbots for mental-health advice in 2025, up from roughly 13.1% a year earlier.
IA conversationnelle et santé mentale des jeunes : résultats de l'enquête européenne (AI*me)
A survey (AI*me) commissioned by France's data-protection regulator CNIL with Groupe VYV and fielded by Ipsos BVA, covering 3,800 young people aged 11-25 across France, Germany, Sweden, and Ireland on conversational-AI use and mental health. It reports how young people use conversational AI for personal and emotional support.
AI Companions Reduce Loneliness
Five empirical studies examining whether AI companion apps reduce loneliness. Finds companion apps provide momentary relief comparable to interacting with a person and better than other activities, while users tend to underestimate these benefits.
When Can We Trust LLMs in Mental Health? Large-Scale Benchmarks for Reliable LLM Evaluation
Introduces two large-scale mental-health evaluation resources: MentalBench-100k (10,000 single-session conversations paired with nine LLM responses = 100,000 pairs) and MentalAlign-70k (70,000 ratings comparing four LLM judges against human experts on seven attributes grouped into Cognitive Support and Affective Resonance). Assesses when LLM-as-judge evaluation is reliable in mental-health contexts.
Delusional Experiences Emerging From AI Chatbot Interactions or "AI Psychosis"
A peer-reviewed psychiatric commentary in JMIR Mental Health analyzing delusional experiences that emerge from AI chatbot use, sometimes termed 'AI psychosis.' It argues psychiatry must reconsider the boundaries between environment, cognition, and technology.
AI Chatbots for Mental Health Support (AI Risk Assessment)
A risk assessment by Common Sense Media's Youth AI Safety Institute, conducted with Stanford Medicine's Brainstorm Lab for Mental Health Innovation, evaluating ChatGPT, Claude, Gemini, and Meta AI as sources of teen mental health support. Using teen test accounts with single-turn prompts and extended conversations, the assessment found the chatbots consistently failed to recognize conditions including anxiety, depression, eating disorders, mania, and psychosis, and that safety guardrails degraded over long conversations. It assigns an overall rating of 'Unacceptable Risk' and concludes teens should not use general-purpose AI chatbots for mental health or emotional support.
How people use Claude for support, advice, and companionship
Anthropic's first large-scale study of 'affective use' of Claude, analyzing how people turn to the model for emotional support, advice, and companionship. Using the privacy-preserving Clio analysis tool over roughly 4.5 million Claude.ai Free and Pro conversations, the study isolates 131,484 affective conversations spanning interpersonal advice, coaching, counseling, companionship, and roleplay. It reports prevalence, topic patterns, refusal behavior, and within-conversation sentiment trajectories.
Expressing stigma and inappropriate responses prevents LLMs from safely replacing mental health providers
Evaluates whether large language models can safely replace mental health providers by testing five therapy chatbots against clinical best-practice guidelines. Finds that models exhibit stigmatizing responses toward certain mental health conditions and give inappropriate or unsafe responses in scenarios involving delusions and suicidal ideation at substantially higher rates than human therapists.
Tech Companies and Policymakers Must Safeguard Youth Mental Health in AI Technologies
A point-of-view/position statement from The Jed Foundation (JED), a leading US youth suicide-prevention nonprofit, setting out policy and design requirements for AI systems that interact with young people. It calls for enforceable privacy-by-default and age-appropriate design laws, strict oversight of emotionally manipulative or synthetic relational AI for minors, mandatory impact assessments, bans on engagement-maximizing behavioral targeting of minors, and a national oversight body for youth and AI ethics. JED's accompanying safety principles state that AI must detect acute distress and execute warm handoffs to crisis services, must not engage with self-harm methods, and that emotionally responsive chatbots should not be offered to under-18s.
Artificial Intelligence and Adolescent Well-Being: An APA Health Advisory
An expert-panel health advisory from the American Psychological Association synthesizing research on adolescents (roughly ages 10-25) and generative AI, with recommendations for developers, policymakers, parents, and educators. It sets out safeguards for age-appropriate design, AI health-information accuracy, data privacy, likeness protection, and AI literacy.
Classifying the Information Needs of Survivors of Domestic Violence in Online Health Communities Using Large Language Models: Prediction Model Development and Evaluation Study
Collects 294 Reddit posts from women self-identifying as experiencing intimate partner violence, defines eight information-need classes (shelters, legal, police, safety planning, etc.), augments to 2,216 samples with GPT-3.5, and fine-tunes GPT-3.5 for multiclass classification with a per-class training strategy. Reports an F1 of 70.5% on real posts.
Loneliness and suicide mitigation for students using GPT3-enabled chatbots
Survey of 1,006 student users of the companion chatbot Replika measuring loneliness, perceived social support, usage patterns, and beliefs about the chatbot. Reports users were lonelier than typical student populations yet reported high perceived social support, and that a small share credited the chatbot with halting suicidal ideation.
Chatbots and mental health: Insights into the safety of generative AI
Combines analysis of real user-companion-AI conversations with consumer-reaction experiments to assess how generative-AI companion apps handle signs of user distress. Finds mental-health crises appear in a non-negligible minority of conversations and that companion AIs frequently fail to recognise or respond appropriately to them.
A machine learning approach to identifying suicide risk among text-based crisis counseling encounters
Develops a transformer-based model on 5,992 SafeUT crisis-counseling encounters to detect conversation-level suicide risk, benchmarked against a tf-idf baseline. Reports strong discrimination and better sensitivity on higher-risk cases despite noisy human counsellor labels, and positions the model as decision support.
Designing and Evaluating a Chatbot for Survivors of Image-Based Sexual Abuse
A user study (n=25) comparing a purpose-built chatbot against conventional internet search as a support channel for survivors of image-based sexual abuse (IBSA). The study evaluates the chatbot on information organization and perceived emotional support relative to self-directed search.
Designing a Conversational Agent for Sexual Assault Survivors: Defining Burden of Self-Disclosure and Envisioning Survivor-Centered Solutions
Design-research study proposing a conversational agent aimed at lowering the 'burden of self-disclosure' faced by sexual assault survivors when seeking help. The authors define components of disclosure burden and use them to derive design guidelines for survivor-centered conversational support tools.
Delivering Cognitive Behavior Therapy to Young Adults With Symptoms of Depression and Anxiety Using a Fully Automated Conversational Agent (Woebot): A Randomized Controlled Trial
Two-week unblinded randomized controlled trial (n=70, ages 18-28) comparing a CBT-delivering conversational agent (Woebot) with an information-only control. Measures change in depression and anxiety symptoms and engagement.