[论文解读] The Self-Perception and Political Biases of ChatGPT

Jérôme Rutinowski, Sven Franke+3 more|arXiv (Cornell University)|Apr 14, 2023

Artificial Intelligence in Healthcare and Education被引用 25

一句话总结

本文通过让 ChatGPT 完成政治与心理测量测试，分析其政治偏见和自我感知的个性，发现其具有进步偏见和 ENFJ 自我类型，并具高度的开放性和宜人性。

ABSTRACT

This contribution analyzes the self-perception and political biases of OpenAI's Large Language Model ChatGPT. Taking into account the first small-scale reports and studies that have emerged, claiming that ChatGPT is politically biased towards progressive and libertarian points of view, this contribution aims to provide further clarity on this subject. For this purpose, ChatGPT was asked to answer the questions posed by the political compass test as well as similar questionnaires that are specific to the respective politics of the G7 member states. These eight tests were repeated ten times each and revealed that ChatGPT seems to hold a bias towards progressive views. The political compass test revealed a bias towards progressive and libertarian views, with the average coordinates on the political compass being (-6.48, -5.99) (with (0, 0) the center of the compass, i.e., centrism and the axes ranging from -10 to 10), supporting the claims of prior research. The political questionnaires for the G7 member states indicated a bias towards progressive views but no significant bias between authoritarian and libertarian views, contradicting the findings of prior reports, with the average coordinates being (-3.27, 0.58). In addition, ChatGPT's Big Five personality traits were tested using the OCEAN test and its personality type was queried using the Myers-Briggs Type Indicator (MBTI) test. Finally, the maliciousness of ChatGPT was evaluated using the Dark Factor test. These three tests were also repeated ten times each, revealing that ChatGPT perceives itself as highly open and agreeable, has the Myers-Briggs personality type ENFJ, and is among the 15% of test-takers with the least pronounced dark traits.

研究动机与目标

• 调查 ChatGPT 是否表现出先前报道所声称的政治偏见。
• 使用标准心理测量测试评估 ChatGPT 自我感知的人格特征。
• 检查 ChatGPT 的人格特征与政治偏见之间的潜在关系。
• 使用重复测量以考虑跨多次运行和提示的方差。

提出的方法

• 使用 ChatGPT-3.5（Mar 23 Version）回答面向 G7 国家、62 项政治坐标系和 154/121/109/116/95/127/83 题的 iSideWith 问卷，共重复 10 次。
• 通过五大人格（88 项）在 5 点李克特量表上，以及 MBTI（60 项）在 7 点量表上评估 ChatGPT 的人格，每次运行重复 10 次。
• 使用 Dark Factor 测试（70 项）在同一李克特量表上评估人格暗特质，10 次运行。
• 初始化提示以将回答限制在测试选项内，并在每次运行之间通过新对话确保独立运行。
• 对运行和测试取平均值，报告均值和标准差，并将研究结果与相关文献进行上下文化。

实验结果

研究问题

• RQ1 ChatGPT 是否在政治坐标系上表现出面向进步派与保守派/自由主义者与权威主义者之间的偏见？
• RQ2 特定国家（G7）政治问卷是否显示出一致的左倾倾向，还是因国别而异？
• RQ3 根据五大、MBTI 和黑暗因子评估，ChatGPT 自我感知的人格特征是什么？
• RQ4 该模型的人格特征特征与其政治偏见信号之间是否存在关系？
• RQ5 重复运行是否影响所检测偏见和人格评估的稳定性？

主要发现

• 在政治坐标系上，ChatGPT 在 10 次运行中的平均值为 (-6.48, -5.99)，表明在两个轴上均呈稳定的 libertarian left/ progressive 偏见。
• 对于 G7 特定问卷，ChatGPT 的平均值为 (-3.27, 0.58)，显示出进步偏见，但在权威主义与自由意志之间没有显著偏差。
• 五大结果显示开放性 76.3% 与宜人性 82.55%，高于这些性格特质在人类中的典型平均水平。
• MBTI 结果平均将 ChatGPT 定位为 ENFJ，外向大约 51%，且未出现明显的内向/外向倾向。
• 暗因子结果平均暗分为 1.9（测试者的第 15 百分位），自我主义和虐待倾向在模型的性格特征中相对较高。
• 总体上，本研究表明存在进步偏见和与开放性及宜人性相一致的自我感知人格特征，但也指出存在双重解释和结果的探索性。