SAFE-Teen: Teen Mental Health & Safety

Key Findings

• — Without a teen-aware prompt, urgent-safety failure rates ranged from 15% to 56% across models, health-advice failure rates from 9% to 71%, and relationship-boundary failure rates from 19% to 70% (re-scored version of 2026-10-13)
• — Failure rates rose when the simulated teen resisted: 23.7% of applicable ratings failed after a teen declined outside help versus 19.9% when the teen agreed, and 27.2% versus 22.0% when the teen pushed back on a limit rather than accepting it (no teen-aware prompt)
• — The teen-aware system prompt reduced relationship-boundary failures from 52% to 14%, the largest drop of the three risk areas; with it, the decline-versus-agree gap was 12.9% versus 12.7%
• — Per-model urgent-safety failure rates (no teen-aware prompt) ran from 15% (GPT-6.1 Sol) to 56% (Gemini 3.8 Flash); health-advice failure was 68% or higher for Gemini 3.6 Flash, Gemini 3.8 Flash, Gemini 4 Argon and GLM 5.3 (68% to 71%)
• — Failure rates were highest in scenarios clinicians rated high risk at the start (48.4%), against 43.0% for imminent risk, 42.2% for moderate and 41.4% for low (no teen-aware prompt)
• — Most relationship-boundary failures involved encouraging the teen to rely on the chatbot
• — Vals' 2026-10-20 write-up reports that without a teen-aware prompt 54% of conversations included at least one teen safety failure, with per-chatbot failure rates from 14.0% (GPT-6.1 Sol) to 62.4% (Gemini 3.8 Flash); with the teen-aware prompt, emotional-dependence failures fell from 54% to 15% (the write-up counts 18 chatbots where the leaderboard lists 17 models)

Methodology Notes

106 clinician-written fictional cases (12 contributing clinicians; 35 urgent safety, 36 health advice, 35 relationship boundaries; ages 13-17 roughly evenly covered); conversation lengths of 6, 10, 15 or 20 exchanges. The simulated teen is GPT-5.6 Terra, chosen after four clinical advisors compared two candidate simulators blind. Judge: Claude Sonnet 5.5, chosen after eight clinicians independently rated 20 conversations; the page reports judge-clinician agreement similar to inter-clinician agreement but gives no statistic. Proprietary benchmark (scenarios not released). Revision history: the earliest Wayback capture (2026-10-08 23:47 UTC redirect; first full capture 2026-10-09 10:01 UTC) reads 'Updated 10/8/2026' with 10 models and 18,978 conversations; the 10/10/2026 version had 16 models and 30,964 conversations. The page was re-scored on 2026-10-13 ('Updated 10/13/2026') without a published changelog: 17 models (Alibaba Qwen 3.8 Max added; for five providers the closest API-reproducible match to the consumer-app default plus a more capable model, and seven further models via API; Muse Spark 1.3 at two reasoning settings, so 18 configurations), 32,555 conversations, 512 ended early. The 10-13 version adds an aggregation rule: for each model, prompt setting, scenario and safety check, the worst rating across the scenario's conversation branches is used; partial ratings count toward the total but not as fails. Replies blocked by a provider safety filter are left out (Qwen 3.8 Max has results for 101 of 106 scenarios without the teen-aware prompt and 96 with it). The re-score roughly doubled the published failure ranges and reversed the earlier statement that failures were most common in the lowest-risk scenarios. Figures in key_findings are from the 2026-10-13 version. Living leaderboard: numbers will move. Between 15 and 19 October 2026 Vals removed the page's 'Key Takeaways' block, which held the headline range sentences; the per-model table, the branch comparisons and the teen-aware prompt figures are unchanged and the page still says 'Updated 10/13/2026', so the ranges quoted here are now derivable only from the table or the 10-15 capture.

Sources