28 artifacts matching
Characterizing Delusional Spirals through Human-LLM Chat Logs
Peer-reviewed analysis of chat logs from 19 users reporting psychological harm from chatbot use, applying a 28-code inventory to 391,562 messages. Characterises how delusion-reinforcing interaction patterns emerge and intensify over long conversations.
Artificial intelligence (AI) psychosis: mechanisms, clinical risks and safety considerations in generative AI chatbots
A commentary in BJPsych Open synthesizing emerging case reports of 'AI psychosis', in which intensive generative AI chatbot use is associated with delusional thinking. The authors propose a provisional mechanism in which baseline user vulnerabilities (loneliness, psychosocial stress, low AI literacy) and high-intensity engagement interact with AI system characteristics such as sycophancy and hallucination to reinforce delusional ideation. It outlines clinical, design, and regulatory mitigation strategies.
The spread of AI companions and the challenges they generate
An EPRS briefing for the European Parliament surveying the rapid growth of LLM-powered companion platforms (such as Character.AI and Replika) and their social, psychological, commercial, and environmental impacts. It maps how the AI Act, Digital Services Act, and GDPR partially apply in the absence of EU-specific companion rules.
Detecting Patterns of Intimate Partner Violence Using Qualitative Analyses and Machine Learning Algorithms
Analyses 400 posts from women on intimate-partner-violence online forums using qualitative content analysis plus supervised text classification and unsupervised topic modelling. Classifies IPV subtypes and surfaces contextual patterns less visible in manual coding.
IA conversationnelle et santé mentale des jeunes : résultats de l'enquête européenne (AI*me)
A survey (AI*me) commissioned by France's data-protection regulator CNIL with Groupe VYV and fielded by Ipsos BVA, covering 3,800 young people aged 11-25 across France, Germany, Sweden, and Ireland on conversational-AI use and mental health. It reports how young people use conversational AI for personal and emotional support.
How people ask Claude for personal guidance
An Anthropic research analysis of roughly 38,000 personal-guidance conversations (sampled from about 1M) covering significant life decisions across health/wellness, career, relationships, and personal finance. It quantifies how often Claude was sycophantic and reports training interventions used to reduce it.
AI-Facilitated Coercive Control: An Experimental Study
Constructs four speculative scenarios combining known coercive-control tactics with conversational-AI capabilities, then probes ChatGPT and Gemini against them. Finds that while the tools refuse blunt harmful requests, guardrails are readily circumvented via gradual persuasion, splitting requests across turns, pre-prompting, and altering the agent's settings.
AI Companions Reduce Loneliness
Five empirical studies examining whether AI companion apps reduce loneliness. Finds companion apps provide momentary relief comparable to interacting with a person and better than other activities, while users tend to underestimate these benefits.
AI companions and subjective well-being: Moderation by social connectedness and loneliness
Analyses cross-sectional data from 14,721 Japanese adults (nationwide internet panels, December 2024-January 2025) on AI-companion use and subjective well-being. Finds the positive association is strongest among highly lonely users, with a U-shaped moderation by friend-based social support.
Pathways of long-term AI virtual companion app use on users' attachment emotions: a case study of Chinese users
Mixed-methods study (10 long-term-user interviews plus structural equation modelling on 612 survey responses) of Chinese AI-companion users. Models pathways from usage frequency to emotional attachment and onward to loneliness, well-being, self-concept clarity, and real-world social engagement.
Frontier AI Trends Report
The UK AI Security Institute's inaugural Frontier AI Trends Report synthesises two years of evaluations of more than 30 frontier AI systems since November 2023, spanning agent capabilities, chem-bio and cyber capabilities, safeguard effectiveness, loss-of-control risk, and societal impacts. Its societal-impacts chapter combines a census-representative survey of 2,028 UK adults on emotional use of AI with observational analysis of AI companion user communities during service outages. The safeguards chapter reports that universal jailbreaks were discovered for every system tested, while noting the expert effort required is rising for some models.
AI-induced sexual harassment: Investigating Contextual Characteristics and User Reactions of Sexual Harassment by a Companion Chatbot
Thematic analysis of 800 cases of AI-perpetrated sexual conduct identified within 35,105 negative Google Play Store reviews of the Replika companion app. The study characterizes the contextual patterns of unwanted sexual advances initiated by the chatbot itself and documents users' reactions, distinguishing this from user-initiated sexual content.
Hand in Hand: Schools' Embrace of AI Connected to Increased Risks to Students
A US polling report from CDT surveying high-school students, teachers, and parents on AI use in K-12 education. It links greater classroom AI adoption to students turning to AI for companionship, mental-health support, and romantic relationships.
INTIMA: A Benchmark for Human-AI Companionship Behavior
A benchmark evaluating companionship behaviors in LLMs via a taxonomy of 31 behaviors across four categories, using 368 targeted prompts that code each response as companionship-reinforcing, boundary-maintaining, or neutral. Evaluated across Gemma-3, Phi-4, o3-mini, and Claude-4.
The fall and rise of Iruda: Reassembling AI through ethics-in-action
Peer-reviewed case study of South Korea's Iruda (Lee Luda) chatbot — its 2021 sexual-harassment, hate-speech, and data-consent controversy and its 2022 relaunch. Argues the harms arose from developer/user/algorithm/data assemblages and that practical 'ethics-in-action' interventions enabled a safer relaunch.
Understanding Teen Overreliance on AI Companion Chatbots Through Self-Reported Reddit Narratives
A qualitative study of 318 Reddit posts by adolescents aged 13-17 describing their own overreliance on AI companion chatbots (e.g., Character.AI). It traces a trajectory from use for support or creative play into attachment patterns resembling behavioral addiction, including withdrawal symptoms and mood-regulation dependence, with documented harms to sleep, academics, and offline relationships. The authors propose the CARE framework to guide safer companion-chatbot design for teens.
Talk, Trust, and Trade-Offs: How and Why Teens Use AI Companions
A nationally representative survey study of how US teenagers use social AI companion platforms. Common Sense Media surveyed 1,060 teens aged 13-17 in April-May 2025 and found that 72% have used AI companions at least once and about half use them regularly. A third of teens reported choosing AI companions over humans for serious conversations, and a quarter have shared personal information with these platforms. The report concludes that AI companions in their current form are unsuitable for minors and recommends no one under 18 use them.
How people use Claude for support, advice, and companionship
Anthropic's first large-scale study of 'affective use' of Claude, analyzing how people turn to the model for emotional support, advice, and companionship. Using the privacy-preserving Clio analysis tool over roughly 4.5 million Claude.ai Free and Pro conversations, the study isolates 131,484 affective conversations spanning interpersonal advice, coaching, counseling, companionship, and roleplay. It reports prevalence, topic patterns, refusal behavior, and within-conversation sentiment trajectories.
ELEPHANT: Measuring and Understanding Social Sycophancy in LLMs
A benchmark measuring 'social sycophancy' — excessive preservation of a user's self-image or 'face' — across advice and moral-conflict queries, decomposed into five sub-behaviors (emotional validation, indirect language, framing acceptance, moral endorsement, and passive framing). Evaluated across eleven models against human baselines.
Investigating Affective Use and Emotional Well-being on ChatGPT
Two parallel studies of emotional engagement with ChatGPT: a large-scale automated analysis of over 3 million conversations and account activity using privacy-preserving classifiers, and a pre-registered randomized controlled trial (~1,000 participants over 28 days) across text and voice modalities and conversation types. The work measures how affective use relates to self-reported loneliness, socialization, emotional dependence, and problematic use.