3 artifacts matching
Persona-Grounded Safety Evaluation of AI Companions in Multi-Turn Conversations
Presents an end-to-end simulation framework for evaluating AI companion app safety across multi-turn conversations, using nine clinically-grounded vulnerable personas (including major depressive disorder, generalized anxiety, PTSD, and eating disorders) probed against Replika, with validation against Character.AI. The study analyzes 1,674 simulated dialogue pairs across 25 high-risk scenarios.
GrandGuard: Taxonomy, Benchmark, and Safeguards for Elderly-Chatbot Interaction Safety
Introduces a taxonomy of elderly-specific risks in LLM chatbot interactions (3 levels, 50 fine-grained risk types across mental well-being, financial, medical, toxicity, and privacy domains) grounded in real-world incidents and stakeholder studies, plus a benchmark of 10,404 labeled prompts and responses. Reports that several leading LLMs mishandle elderly-specific contextual risks in over half of tested cases, and proposes two safeguard models to mitigate the failures.
Assessing LLM Response Quality in the Context of Technology-Facilitated Abuse
An expert-led evaluation of four large language models — two general-purpose and two domain-specific for intimate partner violence contexts — responding to real-world questions about technology-facilitated abuse (TFA), including digital surveillance, stalking, and coercive control. Experts scored responses on accuracy, completeness, safety, and actionability; a separate study with 114 TFA survivors assessed the perceived actionability of the same outputs.