Benchmark / dataset Credible

Pluralistic Behavior Suite: Stress-Testing Multi-Turn Adherence to Custom Behavioral Policies

Introduces the Pluralistic Behavior Suite, a benchmark that stress-tests how well language models keep to custom behavioral policies across multi-turn conversations. It spans 300 custom policies across 30 industries and applies multi-turn adversarial pressure to each. Reported at a NeurIPS 2025 workshop.

Read at arxiv.org Archived copy

Publisher

arXiv (NVIDIA)

Published

7 Nov 2025

Added

today

DOI

—

Key Findings

Tests adherence to 300 developer-defined behavioral policies across 30 industries
Adherence holds under 4% failure in single-turn settings but degrades to as much as 84% failure under multi-turn adversarial pressure
Concludes that current alignment and moderation methods do not reliably enforce custom behavioral policies over extended conversation

Methodology Notes

Preprint (arXiv 2511.05018), NVIDIA author team; accepted to the Multi-Turn Interactions workshop at NeurIPS 2025. Policies are largely corporate, brand, and regulatory rather than crisis-specific.

Sources

arXiv preprint (primary)

Archived snapshot (Wayback Machine) — preserved against link rot

Topics

Eval methodology Benchmarks Model behavior Guardrails & moderation

Authors

Prasoon Varshney, Makesh Narsimhan Sreedhar, Liwei Jiang, Traian Rebedea, Christopher Parisien

Cite This

APA

Prasoon Varshney et al. (2025). Pluralistic Behavior Suite: Stress-Testing Multi-Turn Adherence to Custom Behavioral Policies. arXiv (NVIDIA). https://arxiv.org/abs/2511.05018