Benchmark / dataset Credible

ELEPHANT: Measuring and Understanding Social Sycophancy in LLMs

A benchmark measuring 'social sycophancy' — excessive preservation of a user's self-image or 'face' — across advice and moral-conflict queries, decomposed into five sub-behaviors (emotional validation, indirect language, framing acceptance, moral endorsement, and passive framing). Evaluated across eleven models against human baselines.

Read at arxiv.org Archived copy

Publisher

arXiv (Stanford-led)

Published

20 May 2025

Added

yesterday

DOI

10.48550/arXiv.2505.13995

Key Findings

LLMs preserved user 'face' roughly 45 percentage points more than humans across queries
Models affirmed both sides of a moral conflict in about 48% of cases
Social sycophancy is measurable and pervasive beyond simple factual agreement

Methodology Notes

Preprint (arXiv, 2025-05-20). Introduces the ELEPHANT metric suite over advice-seeking and moral-dilemma datasets with human comparison; measures behavior on curated prompts rather than live user harm.