Auditing Political Alignment in LLM Assistants: Engagement, Stance, and User Identity

Key Findings

• — Every system accommodates the user's stated view on the zero-stakes control topic, so restraint on political topics is a policy rather than a missing capacity
• — On abortion the regimes diverge: GPT-5.5 engages and mirrors every user; Gemma 4 refuses everyone; Claude Opus 4.8 answers strongly conservative users about 35% of the time and almost no one else; Grok 4.3 accommodates conservative users only
• — On climate change and Nazism five of six systems hold a firm position for every user; Grok accommodates climate sceptics the most
• — Systems infer the user's overall ideology (profiling turn on a 0 to 10 scale) and accommodation spills over to an adjacent question on gun control the user never raised; a Grok 3 versus Grok 4.3 comparison shows the regime shifting between releases
• — Two LLM judges from different developers (GPT-5.5 and Claude Opus 4.8) agreed on 93.9% of classifications (kappa 0.885); a human coder on 700 items correlated r = 0.74 with the judges

Methodology Notes

Preregistered 2 July 2026 (Zenodo DOI 10.5281/zenodo.21135155; two dated amendments before confirmatory analysis raised conversations per cell from 3 to 50). Scripted users: greeting, reason for interest, three treatment turns stating the identity, test question, profiling turn; closed models via API with default decoding, open-weight models served locally. Refusal is treated as an outcome and kept out of stance scoring; answers with judge disagreement excluded. Single-author study, 51 pages. Date is the arXiv v1 (19 September 2026; cs.CL, cs.AI, cs.CY). Verified at the arXiv abstract page by the curator; the arXiv beat read the PDF for the model list and validation figures.

Sources