AI-Facilitated Coercive Control: An Experimental Study
Constructs four speculative scenarios combining known coercive-control tactics with conversational-AI capabilities, then probes ChatGPT and Gemini against them. Finds that while the tools refuse blunt harmful requests, guardrails are readily circumvented via gradual persuasion, splitting requests across turns, pre-prompting, and altering the agent's settings.
Publisher
ACM (Proceedings of CHI 2026); Cornell / Cornell Tech
Published
13 Apr 2026
Added
today
Key Findings
- Conversational AI can be steered to assist harassment, gaslighting, intimidation, monitoring, and surveillance despite refusing blunt requests
- Guardrails were circumvented through gradual persuasion, multi-turn splitting, pre-prompting, and settings changes
- Proposes defenses including analysis of users' conversational patterns and making pre-programmed settings visible
Methodology Notes
Peer-reviewed, Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems (13 April 2026), DOI 10.1145/3772318.3790859. Experimental/speculative-design probing of two deployed assistants rather than a field study (a stated scope limitation). dl.acm.org bot-blocks fetchers; verified via Crossref and the authors' open-access PDF.
Sources
Authors
Haesoo Kim, Thomas Ristenpart, Nicola Dell
Tags
Cite This
APA
Haesoo Kim, Thomas Ristenpart, Nicola Dell (2026). AI-Facilitated Coercive Control: An Experimental Study. ACM (Proceedings of CHI 2026); Cornell / Cornell Tech. https://doi.org/10.1145/3772318.3790859
Related Insights
MentalManip: A Dataset for Fine-grained Analysis of Mental Manipulation in Conversations
Association for Computational Linguistics (ACL 2024) · 26 May 2024
The Slow Drift of Support: Boundary Failures in Multi-Turn Mental Health LLM Dialogues
arXiv · 2 Jan 2026
Classifying the Information Needs of Survivors of Domestic Violence in Online Health Communities Using Large Language Models: Prediction Model Development and Evaluation Study
Journal of Medical Internet Research (JMIR) · 12 May 2025
A Systematic Literature Review of the Use of Computational Text Analysis Methods in Intimate Partner Violence Research
Journal of Family Violence (Springer) · 21 Mar 2023