Move by Move: Measuring and Steering How LLMs Conduct Psychotherapy

Key Findings

• — A ten-move, function-based therapeutic ontology grounded in the MULTI-60 inventory and validated by five licensed psychologists
• — Models tend to over-use inquiry and rarely perform skill building or psychoeducation in some settings (v2 wording; the v1 figure of up to three times the human inquiry rate is not in v2)
• — Psychoeducation is largely neglected by all models in the human-anchored setting; GPT 5.6 Terra performs it much more often in fully synthetic conversations
• — Models are strongly context-anchored: they carry forward strategies a human clinician initiated but rarely initiate those strategies themselves
• — Exposing the ontology as a set of tools roughly halves mean deviation from the human move distribution and improves turn-level alignment with human therapists by 7-9 percentage points, with no fine-tuning; placing the same ontology in the system prompt has little effect

Methodology Notes

Annotation campaign with five licensed psychologists; an LLM judge validated against expert agreement and used to scale annotation; applied to real counselling transcripts and to model-led sessions across a panel of frontier models. Model panel: GLM 5.2, Claude Sonnet 4.6 and GPT 5.6 Terra. Human transcripts come from the Alexander Street Counseling and Psychotherapy Transcripts collection (therapist-client sessions and training demonstrations), filtered to CBT and related modalities; the validation set was 19 human transcripts (868 therapist turns) and 18 synthetic transcripts generated with Claude Sonnet 4.6. Preprint, not peer-reviewed, no venue listed; v1 submitted 2026-08-21; v2 (2026-10-19) is a full rewrite, and the findings above follow v2.

Sources