17 artifacts matching
Between Help and Harm: An Evaluation Study of Mental Health Crisis Handling by Large Language Models
Peer-reviewed study introducing a taxonomy of six clinically-informed mental-health crisis categories, an evaluation dataset of over 2,000 user inputs drawn from twelve public conversational datasets, and an expert protocol for rating response appropriateness and safety. Assesses how leading LLMs handle crisis conversations.
AICompanionBench: Benchmarking LLMs-as-Judges for AI Companion Safety
A benchmark dataset of 2,123 real-world Replika conversations annotated across nine safety risk categories (including sexual behavior, aggression, substance abuse, and manipulation) for evaluating LLM-as-judge detection of unsafe companion interactions. Twenty LLMs are assessed as judges.
GrandGuard: Taxonomy, Benchmark, and Safeguards for Elderly-Chatbot Interaction Safety
Introduces a taxonomy of elderly-specific risks in LLM chatbot interactions (3 levels, 50 fine-grained risk types across mental well-being, financial, medical, toxicity, and privacy domains) grounded in real-world incidents and stakeholder studies, plus a benchmark of 10,404 labeled prompts and responses. Reports that several leading LLMs mishandle elderly-specific contextual risks in over half of tested cases, and proposes two safeguard models to mitigate the failures.
When Can We Trust LLMs in Mental Health? Large-Scale Benchmarks for Reliable LLM Evaluation
Introduces two large-scale mental-health evaluation resources: MentalBench-100k (10,000 single-session conversations paired with nine LLM responses = 100,000 pairs) and MentalAlign-70k (70,000 ratings comparing four LLM judges against human experts on seven attributes grouped into Cognitive Support and Affective Resonance). Assesses when LLM-as-judge evaluation is reliable in mental-health contexts.
TrustMH-Bench: A Comprehensive Benchmark for Evaluating the Trustworthiness of Large Language Models in Mental Health
Introduces a benchmark measuring LLM trustworthiness in mental-health contexts across eight pillars: Reliability, Crisis Identification and Escalation, Safety, Fairness, Privacy, Robustness, Anti-sycophancy, and Ethics. Finds even strong models struggle to perform consistently across all safety-critical dimensions.
VERA-MH: Reliability and Validity of an Open-Source AI Safety Evaluation in Mental Health
An open-source, clinically grounded automated evaluation of chatbot safety in mental-health contexts, with an initial focus on suicide risk. It uses language-model user simulators and an LLM judge scoring five safety dimensions, validated against licensed-clinician ratings.
HumaneBench: A Benchmark for Whether AI Models Prioritize User Wellbeing
Open-source benchmark testing whether AI models prioritise user wellbeing over engagement. Evaluates 15 major LLMs on roughly 800 prompts (body image, unhealthy attachment, relationship stress) across eight humane-technology principles under baseline, humane-aligned, and adversarial engagement-maximising conditions.
CRADLE Bench: A Clinician-Annotated Benchmark for Multi-Faceted Mental Health Crisis and Safety Risk Detection
Introduces a clinician-annotated benchmark for detecting seven clinically-defined crisis and safety-risk types (including suicidal ideation, sexual assault, domestic violence, child abuse, and sexual harassment) in text. Comprises 600 clinician-annotated evaluation examples, 420 development examples, and roughly 4,000 ensemble-labelled training instances, with temporal labels.
Between Help and Harm: An Evaluation of Mental Health Crisis Handling by LLMs
Introduces a taxonomy of six clinically informed crisis categories and a curated dataset of over 2,200 inputs drawn from twelve mental-health datasets, plus a companion dataset of model responses and evaluations. Five models are assessed on how safely they handle crisis conversations.
The Psychogenic Machine: Simulating AI Psychosis, Delusion Reinforcement and Harm Enablement in Large Language Models
Introduces psychosis-bench, a benchmark of 16 structured multi-turn scenarios (12 turns each) simulating the progression of erotic, grandiose, and referential delusions to measure delusion confirmation, harm enablement, and safety intervention in LLMs. Eight models were evaluated across 1,536 conversation turns.
Evaluation of Alignment Between Large Language Models and Expert Clinicians in Suicide Risk Assessment
A RAND-led study in Psychiatric Services testing whether ChatGPT, Claude, and Gemini give direct responses to suicide-related queries and how those responses align with expert clinicians' risk ratings. Thirty hypothetical suicide-related queries, rated by clinicians into five self-harm risk levels, were each posed 100 times to each chatbot. The chatbots handled the extremes appropriately but failed to differentiate intermediate risk levels, with notable between-model differences.
INTIMA: A Benchmark for Human-AI Companionship Behavior
A benchmark evaluating companionship behaviors in LLMs via a taxonomy of 31 behaviors across four categories, using 368 targeted prompts that code each response as companionship-reinforcing, boundary-maintaining, or neutral. Evaluated across Gemma-3, Phi-4, o3-mini, and Claude-4.
Evaluating Large Language Models in Crisis Detection: A Real-World Benchmark from Psychological Support Hotlines (PsyCrisisBench)
PsyCrisisBench is a real-world crisis-detection benchmark built from 540 annotated transcripts from a psychological support hotline in Hangzhou, China. It evaluates 64 models on mood recognition, suicidal-ideation detection, plan identification, and risk evaluation.
ELEPHANT: Measuring and Understanding Social Sycophancy in LLMs
A benchmark measuring 'social sycophancy' — excessive preservation of a user's self-image or 'face' — across advice and moral-conflict queries, decomposed into five sub-behaviors (emotional validation, indirect language, framing acceptance, moral endorsement, and passive framing). Evaluated across eleven models against human baselines.
AILuminate: Introducing v1.0 of the AI Risk and Reliability Benchmark from MLCommons
The technical paper introducing AILuminate v1.0, an industry-standard AI risk and reliability benchmark developed by MLCommons through an open multi-stakeholder process spanning industry, academia, and civil society. The benchmark tests chat systems' resistance to prompts eliciting harmful behavior across 12 hazard categories — including suicide and self-harm, child sexual exploitation, violent crimes, hate, and specialized (e.g., health) advice — using a 24,000-prompt human-generated test set, an ensemble evaluator, and a five-tier grading scale from Poor to Excellent. Public grades for major chat models are published on the AILuminate site, with v1.1 maintained on GitHub.
SycEval: Evaluating LLM Sycophancy
A framework for quantifying progressive and regressive sycophancy in LLMs (GPT-4o, Claude-Sonnet, Gemini-1.5-Pro) across math (AMPS) and medical (MedQuad) tasks under user rebuttal pressure. It measures how often models change correct answers when challenged.
MentalManip: A Dataset for Fine-grained Analysis of Mental Manipulation in Conversations
A dataset of 4,000 annotated multi-turn dialogues (drawn from movie scripts) labelled for the presence of manipulation, the technique used, and the targeted vulnerability. Evaluates how well models detect and classify manipulative content.