AICompanionBench: Benchmarking LLMs-as-Judges for AI Companion Safety
A benchmark dataset of 2,123 real-world Replika conversations annotated across nine safety risk categories (including sexual behavior, aggression, substance abuse, and manipulation) for evaluating LLM-as-judge detection of unsafe companion interactions. Twenty LLMs are assessed as judges.
Key Findings
- Models detect explicit harmful content well but struggle with nuanced categories such as manipulation
- Judges sometimes over-flag benign companion conversations as harmful
- Provides the first public benchmark of real companion-platform conversations for judge evaluation
Methodology Notes
Preprint (arXiv 2606.04867, 2026-06-03). Real Replika conversations with nine-category safety annotation; evaluates LLM-as-judge frameworks rather than generation safety.
Sources
arXiv abstract (primary)
Archived snapshot (Wayback Machine) — preserved against link rot
Authors
Yanjing Ren, Reza Ebrahimi, TengTeng Ma
Tags
Cite This
APA
Yanjing Ren, Reza Ebrahimi, TengTeng Ma (2026). AICompanionBench: Benchmarking LLMs-as-Judges for AI Companion Safety. arXiv. https://arxiv.org/abs/2606.04867
Related Insights
INTIMA: A Benchmark for Human-AI Companionship Behavior
arXiv (Hugging Face) · 4 Aug 2025
Investigating Affective Use and Emotional Well-being on ChatGPT
OpenAI; MIT Media Lab · 4 Apr 2025
Findings from transparency notices on AI companion apps: October 2025 (non-periodic)
eSafety Commissioner · 24 Mar 2026
When Can We Trust LLMs in Mental Health? Large-Scale Benchmarks for Reliable LLM Evaluation
Association for Computational Linguistics (EACL 2026) · 24 Mar 2026