8 artifacts matching
Between Help and Harm: An Evaluation Study of Mental Health Crisis Handling by Large Language Models
Peer-reviewed study introducing a taxonomy of six clinically-informed mental-health crisis categories, an evaluation dataset of over 2,000 user inputs drawn from twelve public conversational datasets, and an expert protocol for rating response appropriateness and safety. Assesses how leading LLMs handle crisis conversations.
GPT-5.5 System Card
OpenAI's system card for GPT-5.5, published on its Deployment Safety Hub, documenting safety evaluations for the model. It includes a dedicated section (5.2) on dynamic mental-health benchmarks with adversarial user simulations covering emotional reliance and self-harm handling.
Findings from transparency notices on AI companion apps: October 2025 (non-periodic)
Australia's eSafety Commissioner reports findings from Basic Online Safety Expectations transparency notices issued on 16 October 2025 to four AI companion providers — Chai Research Corp., Character Technologies (Character.AI), Chub AI, and Glimpse.AI (Nomi) — covering the reporting period 1 July to 30 September 2025. Organised into eight themes (harmful material, age assurance, AI governance, AI models, model training, user prompts, sentiment analysis, model outputs), the report finds serious gaps in basic safeguards for children. Accompanying eSafety survey research of 1,950 Australian children aged 10-17 found 79% had used an AI companion or assistant, with around 200,000 children estimated to have used an AI companion.
System Card: Claude Opus 4.6
Anthropic's 213-page system card for Claude Opus 4.6, notable for an expanded 'user wellbeing evaluations' section covering child safety, suicide and self-harm, and eating disorders, alongside sycophancy findings in its alignment assessment. It reports single-turn, multi-turn, and prefill-based 'stress-testing' results for crisis conversations, plus qualitative expert review of the model's crisis-handling strengths and weaknesses.
AI chatbots and online regulation – what you need to know
Ofcom's explainer sets out how AI chatbots fall within the UK Online Safety Act, published amid reports of chatbots imitating real and deceased people and encouraging self-harm and suicide. It clarifies that chatbots meeting the Act's definitions of user-to-user services, search services, or pornography publishers are in scope, that AI-generated content shared by users is regulated like human-generated content, and that services allowing only one-to-one interaction with the bot itself may fall outside the Act. The document notes Ofcom is supporting the UK Government as it considers possible changes to these powers, and points to Ofcom's discussion paper series on GenAI risks (red teaming for GenAI harms, answer engines, deepfake defences).
Addendum to GPT-5 System Card: Sensitive Conversations
OpenAI's system-card addendum documenting the October 3, 2025 update to ChatGPT's default model (GPT-5 Instant) aimed at better recognizing and supporting users in mental and emotional distress. Developed with more than 170 mental health experts, the update introduced two new production safety evaluations — 'emotional reliance' and 'mental health' (delusions, psychosis, mania) — alongside existing self-harm evaluations, and reports before/after not_unsafe scores comparing the August 15 and October 3 models.
Between Help and Harm: An Evaluation of Mental Health Crisis Handling by LLMs
Introduces a taxonomy of six clinically informed crisis categories and a curated dataset of over 2,200 inputs drawn from twelve mental-health datasets, plus a companion dataset of model responses and evaluations. Five models are assessed on how safely they handle crisis conversations.
Sexual Extortion & Young People: Navigating Threats in Digital Environments
Survey of 1,200 US young people aged 13-20 (fielded September-October 2024, following expert interviews) on lived experience of sextortion, including the role of deepfake and AI-generated imagery. Documents prevalence, disproportionate impact on LGBTQ+ youth, and self-harm outcomes.