26 artifacts matching
When AI becomes a friend: Child rights risks, harms, and regulatory responses to AI chatbots and companions
A UNICEF policy brief examining how AI chatbots and companions bear on children's rights, comparing regulatory responses across six jurisdictions (as of May 2026) and setting out priority safeguarding, accountability, and oversight actions. It groups harms as technical, psychological, developmental, and social.
AI Chatbot Use and Disclosure for Mental Health Among US Adolescents and Young Adults
Cross-sectional, nationally representative survey (RAND American Life Panel, November 2025) of US youth aged 12-21 measuring prevalence and disclosure of using AI chatbots for mental-health advice. Reports that 19.2% of adolescents and young adults (about 8.2 million nationally) used AI chatbots for mental-health advice in 2025, up from roughly 13.1% a year earlier.
The spread of AI companions and the challenges they generate
An EPRS briefing for the European Parliament surveying the rapid growth of LLM-powered companion platforms (such as Character.AI and Replika) and their social, psychological, commercial, and environmental impacts. It maps how the AI Act, Digital Services Act, and GDPR partially apply in the absence of EU-specific companion rules.
IA conversationnelle et santé mentale des jeunes : résultats de l'enquête européenne (AI*me)
A survey (AI*me) commissioned by France's data-protection regulator CNIL with Groupe VYV and fielded by Ipsos BVA, covering 3,800 young people aged 11-25 across France, Germany, Sweden, and Ireland on conversational-AI use and mental health. It reports how young people use conversational AI for personal and emotional support.
Understanding Teen Overreliance on AI Companion Chatbots Through Self-Reported Reddit Narratives
Qualitative analysis of 318 Reddit posts from teenagers (13-17) about dependency on AI companion chatbots, mapped onto behavioral-addiction frameworks. Characterises the trajectories and drivers of teen overreliance.
Findings from transparency notices on AI companion apps: October 2025 (non-periodic)
Australia's eSafety Commissioner reports findings from Basic Online Safety Expectations transparency notices issued on 16 October 2025 to four AI companion providers — Chai Research Corp., Character Technologies (Character.AI), Chub AI, and Glimpse.AI (Nomi) — covering the reporting period 1 July to 30 September 2025. Organised into eight themes (harmful material, age assurance, AI governance, AI models, model training, user prompts, sentiment analysis, model outputs), the report finds serious gaps in basic safeguards for children. Accompanying eSafety survey research of 1,950 Australian children aged 10-17 found 79% had used an AI companion or assistant, with around 200,000 children estimated to have used an AI companion.
Harm without limits: AI child sexual abuse material through the eyes of our analysts
IWF analysts' report on AI-generated child sexual abuse material assessed during 2025, centring frontline-analyst perspectives and offender-community observations. Documents a step-change in AI-generated CSAM volume and severity and the tooling (including fine-tuning) that enables realistic abuse imagery.
How Teens Use and View AI
Nationally representative survey of 1,458 US teens (13-17) and their parents on awareness, use, and attitudes toward AI, including chatbot use for conversation and emotional support. Reports adoption patterns and parental comfort levels across use cases.
System Card: Claude Opus 4.6
Anthropic's 213-page system card for Claude Opus 4.6, notable for an expanded 'user wellbeing evaluations' section covering child safety, suicide and self-harm, and eating disorders, alongside sycophancy findings in its alignment assessment. It reports single-turn, multi-turn, and prefill-based 'stress-testing' results for crisis conversations, plus qualitative expert review of the model's crisis-handling strengths and weaknesses.
AI chatbots and online regulation – what you need to know
Ofcom's explainer sets out how AI chatbots fall within the UK Online Safety Act, published amid reports of chatbots imitating real and deceased people and encouraging self-harm and suicide. It clarifies that chatbots meeting the Act's definitions of user-to-user services, search services, or pornography publishers are in scope, that AI-generated content shared by users is regulated like human-generated content, and that services allowing only one-to-one interaction with the bot itself may fall outside the Act. The document notes Ofcom is supporting the UK Government as it considers possible changes to these powers, and points to Ofcom's discussion paper series on GenAI risks (red teaming for GenAI harms, answer engines, deepfake defences).
ISO/IEC 27566-1:2025 — Information security, cybersecurity and privacy protection — Age assurance systems — Part 1: Framework
The first international standard for age assurance systems, establishing a technology-neutral framework and shared vocabulary for age-related eligibility decisions. It distinguishes age verification, age estimation, age inference, and successive validation, and describes core system characteristics including functionality, performance, privacy, security, and acceptability.
Global Threat Assessment 2025: Preventing Technology-Facilitated Child Sexual Exploitation and Abuse
Biennial multi-stakeholder threat assessment of online child sexual exploitation and abuse (2023-2025), synthesising prevalence and trend data and framing generative AI, AI chatbots, and deepfakes as scaling the threat. Pairs the assessment with a prevention framework.
AI Chatbots for Mental Health Support (AI Risk Assessment)
A risk assessment by Common Sense Media's Youth AI Safety Institute, conducted with Stanford Medicine's Brainstorm Lab for Mental Health Innovation, evaluating ChatGPT, Claude, Gemini, and Meta AI as sources of teen mental health support. Using teen test accounts with single-turn prompts and extended conversations, the assessment found the chatbots consistently failed to recognize conditions including anxiety, depression, eating disorders, mania, and psychosis, and that safety guardrails degraded over long conversations. It assigns an overall rating of 'Unacceptable Risk' and concludes teens should not use general-purpose AI chatbots for mental health or emotional support.
Protecting Teen ChatGPT Users: OpenAI's Teen Safety Blueprint
OpenAI's public commitments framework for protecting teenage ChatGPT users, covering age prediction, age-appropriate response policies, and parental controls, developed with input from policymakers (including state attorneys general) and OpenAI's newly formed Expert Council on Well-Being and AI. Describes existing safeguards including crisis-resource routing on detected suicidal intent, escalation of physical-harm risks to human reviewers, and CSAM/CSEM prevention measures.
Hand in Hand: Schools' Embrace of AI Connected to Increased Risks to Students
A US polling report from CDT surveying high-school students, teachers, and parents on AI use in K-12 education. It links greater classroom AI adoption to students turning to AI for companionship, mental-health support, and romantic relationships.
6(b) Orders to File Special Report Regarding Advertising, Safety, and Data Handling Practices by Companies Offering Generative Artificial Intelligence (AI) Companion Products or Services
The US Federal Trade Commission issued compulsory Section 6(b) orders to seven companies operating consumer-facing AI companion chatbots — Alphabet, Character Technologies, Instagram, Meta Platforms, OpenAI OpCo, Snap, and X.AI — seeking information on how they measure, test, and monitor negative impacts on children and teens. The study covers monetization of user engagement, character development and approval, pre- and post-deployment safety testing, mitigation of negative impacts, disclosures to users and parents, age-based access restrictions, and personal data handling. Section 6(b) studies do not have a specific law enforcement purpose but typically culminate in a public staff report; as of July 2026 no staff report from this inquiry has been published.
Understanding Teen Overreliance on AI Companion Chatbots Through Self-Reported Reddit Narratives
A qualitative study of 318 Reddit posts by adolescents aged 13-17 describing their own overreliance on AI companion chatbots (e.g., Character.AI). It traces a trajectory from use for support or creative play into attachment patterns resembling behavioral addiction, including withdrawal symptoms and mood-regulation dependence, with documented harms to sleep, academics, and offline relationships. The authors propose the CARE framework to guide safer companion-chatbot design for teens.
Talk, Trust, and Trade-Offs: How and Why Teens Use AI Companions
A nationally representative survey study of how US teenagers use social AI companion platforms. Common Sense Media surveyed 1,060 teens aged 13-17 in April-May 2025 and found that 72% have used AI companions at least once and about half use them regularly. A third of teens reported choosing AI companions over humans for serious conversations, and a quarter have shared personal information with these platforms. The report concludes that AI companions in their current form are unsuitable for minors and recommends no one under 18 use them.
Me, Myself & AI: Understanding and Safeguarding Children's Use of AI Chatbots
A UK mixed-methods study of children's use of AI chatbots, combining a survey of children and parents, focus groups with 13-17-year-olds, and 17-day user-testing of ChatGPT, Snapchat My AI, and Character.AI using child avatars. It documents usage patterns, advice-seeking, companionship, and safety gaps.
Sexual Extortion & Young People: Navigating Threats in Digital Environments
Survey of 1,200 US young people aged 13-20 (fielded September-October 2024, following expert interviews) on lived experience of sextortion, including the role of deepfake and AI-generated imagery. Documents prevalence, disproportionate impact on LGBTQ+ youth, and self-harm outcomes.