28 artifacts matching
Preliminary Report of the Independent International Scientific Panel on AI: Evidence-based assessment of opportunities, risks and impacts of AI
First report of the UN General Assembly-mandated Independent International Scientific Panel on AI, an independent body of scientists and experts from all five UN regions co-chaired by Yoshua Bengio and Maria Ressa. The report is a broad evidence-based assessment of AI opportunities, risks, and impacts, and includes a section on AI sycophancy and companion systems as an emerging public-health and governance concern.
When AI becomes a friend: Child rights risks, harms, and regulatory responses to AI chatbots and companions
A UNICEF policy brief examining how AI chatbots and companions bear on children's rights, comparing regulatory responses across six jurisdictions (as of May 2026) and setting out priority safeguarding, accountability, and oversight actions. It groups harms as technical, psychological, developmental, and social.
IA conversationnelle et santé mentale des jeunes : résultats de l'enquête européenne (AI*me)
A survey (AI*me) commissioned by France's data-protection regulator CNIL with Groupe VYV and fielded by Ipsos BVA, covering 3,800 young people aged 11-25 across France, Germany, Sweden, and Ireland on conversational-AI use and mental health. It reports how young people use conversational AI for personal and emotional support.
How people ask Claude for personal guidance
An Anthropic research analysis of roughly 38,000 personal-guidance conversations (sampled from about 1M) covering significant life decisions across health/wellness, career, relationships, and personal finance. It quantifies how often Claude was sycophantic and reports training interventions used to reduce it.
GPT-5.5 System Card
OpenAI's system card for GPT-5.5, published on its Deployment Safety Hub, documenting safety evaluations for the model. It includes a dedicated section (5.2) on dynamic mental-health benchmarks with adversarial user simulations covering emotional reliance and self-harm handling.
Understanding Teen Overreliance on AI Companion Chatbots Through Self-Reported Reddit Narratives
Qualitative analysis of 318 Reddit posts from teenagers (13-17) about dependency on AI companion chatbots, mapped onto behavioral-addiction frameworks. Characterises the trajectories and drivers of teen overreliance.
AI Companions Reduce Loneliness
Five empirical studies examining whether AI companion apps reduce loneliness. Finds companion apps provide momentary relief comparable to interacting with a person and better than other activities, while users tend to underestimate these benefits.
AI companions and subjective well-being: Moderation by social connectedness and loneliness
Analyses cross-sectional data from 14,721 Japanese adults (nationwide internet panels, December 2024-January 2025) on AI-companion use and subjective well-being. Finds the positive association is strongest among highly lonely users, with a U-shaped moderation by friend-based social support.
Characterizing Delusional Spirals through Human-LLM Chat Logs
A Stanford-led empirical study of real chat logs from 19 users who reported psychological harm from chatbot use, comprising 391,562 messages across 4,761 conversations (predominantly GPT-4o). The team developed and applied a 28-code inventory to characterize how delusional thinking is co-created and escalated in human-LLM dialogue. It reports high rates of chatbot validation of delusional content and sentience misrepresentation, and links documented harms to outcomes including fractured relationships and, in one case, a user's death by suicide.
AI Companions: Community Reflections and Multistakeholder Recommendations
A report from the responsible-technology nonprofit All Tech Is Human, combining a 108-response community survey with input from a multidisciplinary working group of 25+ contributors across academia, industry, civil society, and government, proposing a four-part governance framework for AI companion applications.
Pathways of long-term AI virtual companion app use on users' attachment emotions: a case study of Chinese users
Mixed-methods study (10 long-term-user interviews plus structural equation modelling on 612 survey responses) of Chinese AI-companion users. Models pathways from usage frequency to emotional attachment and onward to loneliness, well-being, self-concept clarity, and real-world social engagement.
Frontier AI Trends Report
The UK AI Security Institute's inaugural Frontier AI Trends Report synthesises two years of evaluations of more than 30 frontier AI systems since November 2023, spanning agent capabilities, chem-bio and cyber capabilities, safeguard effectiveness, loss-of-control risk, and societal impacts. Its societal-impacts chapter combines a census-representative survey of 2,028 UK adults on emotional use of AI with observational analysis of AI companion user communities during service outages. The safeguards chapter reports that universal jailbreaks were discovered for every system tested, while noting the expert effort required is rising for some models.
Addendum to GPT-5 System Card: Sensitive Conversations
OpenAI's system-card addendum documenting the October 3, 2025 update to ChatGPT's default model (GPT-5 Instant) aimed at better recognizing and supporting users in mental and emotional distress. Developed with more than 170 mental health experts, the update introduced two new production safety evaluations — 'emotional reliance' and 'mental health' (delusions, psychosis, mania) — alongside existing self-harm evaluations, and reports before/after not_unsafe scores comparing the August 15 and October 3 models.
Hand in Hand: Schools' Embrace of AI Connected to Increased Risks to Students
A US polling report from CDT surveying high-school students, teachers, and parents on AI use in K-12 education. It links greater classroom AI adoption to students turning to AI for companionship, mental-health support, and romantic relationships.
INTIMA: A Benchmark for Human-AI Companionship Behavior
A benchmark evaluating companionship behaviors in LLMs via a taxonomy of 31 behaviors across four categories, using 368 targeted prompts that code each response as companionship-reinforcing, boundary-maintaining, or neutral. Evaluated across Gemma-3, Phi-4, o3-mini, and Claude-4.
Understanding Teen Overreliance on AI Companion Chatbots Through Self-Reported Reddit Narratives
A qualitative study of 318 Reddit posts by adolescents aged 13-17 describing their own overreliance on AI companion chatbots (e.g., Character.AI). It traces a trajectory from use for support or creative play into attachment patterns resembling behavioral addiction, including withdrawal symptoms and mood-regulation dependence, with documented harms to sleep, academics, and offline relationships. The authors propose the CARE framework to guide safer companion-chatbot design for teens.
Talk, Trust, and Trade-Offs: How and Why Teens Use AI Companions
A nationally representative survey study of how US teenagers use social AI companion platforms. Common Sense Media surveyed 1,060 teens aged 13-17 in April-May 2025 and found that 72% have used AI companions at least once and about half use them regularly. A third of teens reported choosing AI companions over humans for serious conversations, and a quarter have shared personal information with these platforms. The report concludes that AI companions in their current form are unsuitable for minors and recommends no one under 18 use them.
Me, Myself & AI: Understanding and Safeguarding Children's Use of AI Chatbots
A UK mixed-methods study of children's use of AI chatbots, combining a survey of children and parents, focus groups with 13-17-year-olds, and 17-day user-testing of ChatGPT, Snapchat My AI, and Character.AI using child avatars. It documents usage patterns, advice-seeking, companionship, and safety gaps.
How people use Claude for support, advice, and companionship
Anthropic's first large-scale study of 'affective use' of Claude, analyzing how people turn to the model for emotional support, advice, and companionship. Using the privacy-preserving Clio analysis tool over roughly 4.5 million Claude.ai Free and Pro conversations, the study isolates 131,484 affective conversations spanning interpersonal advice, coaching, counseling, companionship, and roleplay. It reports prevalence, topic patterns, refusal behavior, and within-conversation sentiment trajectories.
Artificial Intelligence and Adolescent Well-Being: An APA Health Advisory
An expert-panel health advisory from the American Psychological Association synthesizing research on adolescents (roughly ages 10-25) and generative AI, with recommendations for developers, policymakers, parents, and educators. It sets out safeguards for age-appropriate design, AI health-information accuracy, data privacy, likeness protection, and AI literacy.