[论文解读] Perceived Political Bias in LLMs Reduces Persuasive Abilities

Matthew DiGiuseppe, Joshua Robison|arXiv (Cornell University)|Feb 20, 2026

Misinformation and Its Impacts被引用 0

一句话总结

本研究在美国 preregister 实验中（N=2,144）显示，暗示 ChatGPT 对受访者所属党派存在偏见的简短线索可以使基于大模型的劝导效应下降约 28%。

ABSTRACT

Conversational AI has been proposed as a scalable way to correct public misconceptions and spread misinformation. Yet its effectiveness may depend on perceptions of its political neutrality. As LLMs enter partisan conflict, elites increasingly portray them as ideologically aligned. We test whether these credibility attacks reduce LLM-based persuasion. In a preregistered U.S. survey experiment (N=2144), participants completed a three-round conversation with ChatGPT about a personally held economic policy misconception. Compared to a neutral control, a short message indicating that the LLM was biased against the respondent's party attenuated persuasion by 28%. Transcript analysis indicates that the warnings alter the interaction: respondents push back more and engage less receptively. These findings suggest that the persuasive impact of conversational AI is politically contingent, constrained by perceptions of partisan alignment.

研究动机与目标

• 检验对大模型中立性的认知是否限制其政治劝导作用。
• 在经济政策误解方面测试偏见线索对大模型驱动的信念变化的因果效应。
• 评估接触偏见信息对参与度、信任度以及在政治或政策挑战中使用 AI 的意愿的影响。

提出的方法

• Four-arm between-subjects preregistered U.S. survey experiment (N=2,144).
• Three-round conversation with a non-reasoningChatGPT (model 4.1) about six economic misconceptions.
• Random assignment to treatment arms: control; undirected bias; out-party bias (light); out-party bias (heavy).
• LLM guided with a universal prompt to persuade to the consensus while remaining truthful.
• Transcript analysis using a Bayesian Bradley–Terry model to rate argumentativeness and dismissiveness.
• OLS models with topic fixed effects and pretreatment agreement to estimate post-treatment misconception agreement.

实验结果

研究问题

• RQ1 signaling that an LLM is biased against a respondent’s party reduce its persuasive impact on economic misconceptions?
• RQ2 How large is the attenuation of persuasion due to bias cues, and is it consistent across topics and partisans?
• RQ3 What behavioral and attitudinal consequences (e.g., perceived persuasiveness, trust in AI, willingness to reuse) accompany bias cues?

主要发现

• A heavy bias cue reduces persuasion by 28% relative to control (95% CI: 16%–40%).
• In the control group, the mean pre–post change in misconception agreement was -1.20 on a 0–4 scale; in the heavy treatment group it was -0.86, a 28% attenuation.
• 34.4% of respondents in the control group reversed their views, versus 22.1% in the heavy treatment group.
• Out-party bias cues increase perceived bias and reduce agreement with misconceptions across several topics.
• Respondents in heavy/light treatment arms reported lower perceived persuasiveness of the LLM and reduced willingness to use AI again.
• Transcript analysis showed increased engagement and argumentativeness rather than disengagement under bias cues.