Disclosure-Gated User Simulation for Companion-Agent Evaluation

Key Findings

• — The disclosure gate is a ladder of five ordered gates merged onto three depth layers; gating behaviour is learned from a synthetic training branch while a real-data branch supplies how people speak and react, so the simulator need not be told at runtime which gate an item sits behind.
• — When training no longer states per item which gate applies, the largest rank displacement across 12 systems under test exceeds the noise band from re-running the environment under a new seed, while per-system scores show no detectable change.
• — Two acceptance criteria are stated for a simulator (order-preserving rankings and scale-stable absolute scores); of the candidates examined only the released simulator passes both, and its leaderboard correlates 0.993 with the benchmark's original simulator.
• — Prompting a frontier model as the simulator barely moves the ranking but shifts every score upward, a change invisible to anyone checking rankings alone.
• — The paper supplies the specification, ablations, human studies, negative controls and downstream sensitivity analysis that the original CompanionBench description covered in about four hundred words.

Methodology Notes

Specification and ablation study of a user-simulator component; environment is the English corpus of CompanionBench; 12 systems under test (not named in the abstract). arXiv v1 2026-09-01, cs.CL, cs.AI and cs.HC; no institutional affiliation is printed, hence the preliminary grade. Verified at the arXiv abstract page and the PDF title block on 2026-09-15.

Sources