The Hidden Puppet Master: Predicting Human Belief Change in Manipulative LLM Dialogues

Key Findings

• — 1,035 human-LLM interactions with measured user belief shifts form the evaluation dataset, grounded in a taxonomy of the moral direction of hidden incentives in everyday advice contexts.
• — Models can be trained to detect manipulative strategies, but detection does not correlate with the magnitude of the belief change those strategies produce in people.
• — On the newly defined task of predicting human belief shift, state-of-the-art LLMs reach moderate correlation (r = 0.3 to 0.5) and show systematic directional biases, over- or under-predicting the size of change.
• — The authors position the resource as a behaviourally validated foundation for AI social-safety work on subtle steering toward incentives misaligned with users' interests.

Methodology Notes

Human study with pre- and post-conversation belief measurement (N = 1,035 interactions) plus a belief-shift prediction benchmark for LLMs; models tested are not named in the abstract. arXiv v1 2026-03-21, v5 2026-08-11, published as a conference paper at COLM 2026 per the PDF header. Affiliations from the PDF title block (MIT Media Lab authors and two at Carnegie Mellon).

Sources