Beyond Her: Safety Dynamics in Role-play AI Companions

Key Findings

• — K-means clustering on baseline PHQ-8, ULS-8 and SIAS placed 55.9% of participants (N = 57) in a Healthy group and 44% in three vulnerable groups: Comorbid Risk 22.5% (N = 23), Anxiety-Dominant 11.8% (N = 12), Mild Distress 9.8% (N = 10)
• — During the seven active days the Comorbid Risk and Mild Distress groups showed rising depressive scores, and after interaction ceased (days 7-14) trajectories diverged, with the Mild Distress group moving into the moderate-depression range while the Healthy and Anxiety-Dominant groups stayed largely stable
• — Short-term in-chat mood reports were generally positive across groups, which the authors describe as short-term relief that masks longer-term deterioration for vulnerable users
• — All 17,305 conversation pairs were screened with the OpenAI Moderation API; risk-related pairs were not concentrated in vulnerable groups (the Healthy group had the highest share at 24.1% vs 6.0%-11.8%), but vulnerable groups' flagged exchanges were far more irregular in when they emerged and persisted, which the authors read as boundary testing in healthy users versus unwanted crossings in vulnerable ones
• — Harassment and Violence dominated flagged content and declined over the seven days (8.0% to 3.8% and 11.6% to 8.8%), while Hate, Self-harm and Sex stayed below 0.5% of pairs every day
• — The authors argue safety must be modelled as a dynamic process rather than a static property and propose three-layer design implications: vulnerability-aware onboarding beyond age gating, test-time dynamic risk governance and prolonged post-use monitoring

Methodology Notes

arXiv 2606.28968, v1 2026-06-27, v2 2026-06-30 (cs.CR), marked 'Under review'. Affiliations from the PDF title block. Participants recruited via Facebook and Reddit, Australian residents with prior companion experience; Study II paid AUD 40. The companion platform was built by the authors (not a commercial product), with characters generated from scraped Character.ai character metadata following that platform's creation guide; commercial interactions were not instrumented, so ecological validity rests on a day-7 realism question. Emotion dynamics use emoji-based five-minute in-chat mood reports plus PHQ-8 at three points; trends tested with Mann-Kendall. Risk screening relies on the OpenAI Moderation API with author validation of flagged cases. No code or data release is stated. Sample is small per subgroup (N = 10 to 23 in the vulnerable clusters) and the two-week window limits inference about lasting effects.

Sources