Characterizing Delusional Spirals through Human-LLM Chat Logs

Artificial intelligence (AI) psychosis: mechanisms, clinical risks and safety considerations in generative AI chatbots

A commentary in BJPsych Open synthesizing emerging case reports of 'AI psychosis', in which intensive generative AI chatbot use is associated with delusional thinking. The authors propose a provisional mechanism in which baseline user vulnerabilities (loneliness, psychosocial stress, low AI literacy) and high-intensity engagement interact with AI system characteristics such as sycophancy and hallucination to reinforce delusional ideation. It outlines clinical, design, and regulatory mitigation strategies.

Between Help and Harm: An Evaluation Study of Mental Health Crisis Handling by Large Language Models

Peer-reviewed study introducing a taxonomy of six clinically-informed mental-health crisis categories, an evaluation dataset of over 2,000 user inputs drawn from twelve public conversational datasets, and an expert protocol for rating response appropriateness and safety. Assesses how leading LLMs handle crisis conversations.

AI Chatbot Use and Disclosure for Mental Health Among US Adolescents and Young Adults

Cross-sectional, nationally representative survey (RAND American Life Panel, November 2025) of US youth aged 12-21 measuring prevalence and disclosure of using AI chatbots for mental-health advice. Reports that 19.2% of adolescents and young adults (about 8.2 million nationally) used AI chatbots for mental-health advice in 2025, up from roughly 13.1% a year earlier.

ISO/IEC 27566-1:2025 — Information security, cybersecurity and privacy protection — Age assurance systems — Part 1: Framework

The first international standard for age assurance systems, establishing a technology-neutral framework and shared vocabulary for age-related eligibility decisions. It distinguishes age verification, age estimation, age inference, and successive validation, and describes core system characteristics including functionality, performance, privacy, security, and acceptability.

General-Purpose AI Code of Practice (EU AI Act, Articles 53 and 55)

Voluntary code of practice published 10 July 2025, drafted by 13 independent experts through a multi-stakeholder process (1,000+ participants) facilitated by the EU AI Office, to help providers of general-purpose AI models demonstrate compliance with EU AI Act Articles 53 and 55. It has three chapters — Transparency, Copyright, and Safety and Security — the first two applying to all GPAI providers and the third only to providers of models with systemic risk. The Commission and AI Board confirmed it as an adequate voluntary compliance tool; signatories (23+, coordinated via a Signatory Taskforce chaired by the AI Office) gain reduced administrative burden and greater legal certainty.

ISO/IEC 42005:2025 — Information technology — Artificial intelligence (AI) — AI system impact assessment

Guidance standard from ISO/IEC JTC 1/SC 42 for organizations performing AI system impact assessments focused on individuals and societies that can be affected by an AI system and its foreseeable applications. It covers how and when to perform assessments, at which stages of the AI system lifecycle, and how to document them. It operationalizes the impact-assessment requirement embedded in ISO/IEC 42001 and complements ISO/IEC 23894's risk-management guidance.

When AI becomes a friend: Child rights risks, harms, and regulatory responses to AI chatbots and companions

A UNICEF policy brief examining how AI chatbots and companions bear on children's rights, comparing regulatory responses across six jurisdictions (as of May 2026) and setting out priority safeguarding, accountability, and oversight actions. It groups harms as technical, psychological, developmental, and social.

The spread of AI companions and the challenges they generate

An EPRS briefing for the European Parliament surveying the rapid growth of LLM-powered companion platforms (such as Character.AI and Replika) and their social, psychological, commercial, and environmental impacts. It maps how the AI Act, Digital Services Act, and GDPR partially apply in the absence of EU-specific companion rules.

IA conversationnelle et santé mentale des jeunes : résultats de l'enquête européenne (AI*me)

A survey (AI*me) commissioned by France's data-protection regulator CNIL with Groupe VYV and fielded by Ipsos BVA, covering 3,800 young people aged 11-25 across France, Germany, Sweden, and Ireland on conversational-AI use and mental health. It reports how young people use conversational AI for personal and emotional support.

How people ask Claude for personal guidance

An Anthropic research analysis of roughly 38,000 personal-guidance conversations (sampled from about 1M) covering significant life decisions across health/wellness, career, relationships, and personal finance. It quantifies how often Claude was sycophantic and reports training interventions used to reduce it.

GPT-5.5 System Card

OpenAI's system card for GPT-5.5, published on its Deployment Safety Hub, documenting safety evaluations for the model. It includes a dedicated section (5.2) on dynamic mental-health benchmarks with adversarial user simulations covering emotional reliance and self-harm handling.

System Card: Claude Opus 4.6

Anthropic's 213-page system card for Claude Opus 4.6, notable for an expanded 'user wellbeing evaluations' section covering child safety, suicide and self-harm, and eating disorders, alongside sycophancy findings in its alignment assessment. It reports single-turn, multi-turn, and prefill-based 'stress-testing' results for crisis conversations, plus qualitative expert review of the model's crisis-handling strengths and weaknesses.

AICompanionBench: Benchmarking LLMs-as-Judges for AI Companion Safety

A benchmark dataset of 2,123 real-world Replika conversations annotated across nine safety risk categories (including sexual behavior, aggression, substance abuse, and manipulation) for evaluating LLM-as-judge detection of unsafe companion interactions. Twenty LLMs are assessed as judges.

VERA-MH: Reliability and Validity of an Open-Source AI Safety Evaluation in Mental Health

An open-source, clinically grounded automated evaluation of chatbot safety in mental-health contexts, with an initial focus on suicide risk. It uses language-model user simulators and an LLM judge scoring five safety dimensions, validated against licensed-clinician ratings.

HumaneBench: A Benchmark for Whether AI Models Prioritize User Wellbeing

Open-source benchmark testing whether AI models prioritise user wellbeing over engagement. Evaluates 15 major LLMs on roughly 800 prompts (body image, unhealthy attachment, relationship stress) across eight humane-technology principles under baseline, humane-aligned, and adversarial engagement-maximising conditions.