[论文解读] Computational Language Acquisition with Theory of Mind

Andy Liu, Hao Zhu+3 more|arXiv (Cornell University)|Mar 2, 2023

Topic Modeling被引用 5

一句话总结

本文提出了一种计算语言习得框架，其中说话者智能体利用内部心理理论（ToM）听者模型对话语进行重排序，以在图像指代游戏任务中提升语用能力。通过整合能够预测听者在目标选择中信心程度的ToM组件，模型在准确率方面表现更优，且生成的语言更加流畅、精确，尤其在环境难度增加时表现更佳。

ABSTRACT

Unlike current state-of-the-art language models, young children actively acquire language through interactions with their surrounding environment and caretakers. One mechanism that has been argued to be critical to language learning is the ability to infer the mental states of other agents in social environments, coined Theory of Mind (ToM) by Premack & Woodruff (1978). Drawing inspiration from the modern operationalized versions of ToM implemented in Rabinowitz et al. (2018) and Zhu et al. (2021), we build language-learning agents equipped with ToM, and measure its effects on the learning process. We model ToM by giving the speaker agent an internal listener model that is trained alongside the speaker and used to rerank potential utterances. We experiment with varying task difficulty, hypothesizing that models will acquire more complex language to adapt to stronger environmental pressures. We find that training speakers with a highly weighted ToM listener component leads to performance gains in our image referential game setting. We also find some evidence that increasing task difficulty in the training process results in more fluent and precise utterances in evaluation. This suggests the potential utility of further incorporating ToM, as well as other insights from child language acquisition, into computational models of language acquisition.

研究动机与目标

• 探究心理理论（ToM）机制在互动式、具身化环境中对计算智能体语言习得的影响。
• 考察环境难度增加（特别是更具语义和视觉相似性的干扰图像）对所学语言复杂度与质量的影响。
• 通过整合语用推理与社会认知，弥合发展心理学中关于人类语言习得的见解与计算模型之间的鸿沟。
• 评估ToM与环境压力是否共同促进人工智能体生成更类人、流畅且精确的语言。

提出的方法

• 为说话者智能体配备一个内部听者模型，该模型与说话者同步训练，用于预测在给定话语下听者对目标图像选择的信心程度。
• 利用内部ToM听者模型的概率得分对候选话语进行重排序，优先选择在听者预期中最可能成功的话语。
• 在指代游戏设置中训练说话者，使其在多个干扰项中生成英文话语以描述目标图像，奖励基于听者的选择准确率与信心程度。
• 通过基于CLIP、RoBERTa和TF-IDF嵌入的图像表征分布采样干扰项，控制视觉与语义相似性，以调节任务难度。
• 通过调整干扰项相似性，采用课程式训练策略，模拟环境压力的逐步增加。
• 通过目标选择准确率衡量性能，并利用自动指标与人工评估综合评价生成话语的流畅性与精确性。

Figure 1: An example of how ToM is used by our speaker models in our implementation. The speaker first generates candidate utterances before reranking them according to the probability given to each target image, utterance pair by the internal ToM listener. It then selects either the highest-scoring

实验结果

研究问题

• RQ1 RQ1：内部心理理论（ToM）听者组件的引入如何影响语言学习智能体的性能与语言质量？
• RQ2 RQ2：当干扰项图像更加相似时，模型如何调整其语言输出以应对环境难度的提升？
• RQ3 RQ3：ToM与环境压力的结合是否能促成更流畅、更精确、更类人的语言生成？
• RQ4 RQ4：在不同任务条件下，配备ToM的模型在语用语言生成方面相较于非ToM模型的性能优势有多大？

主要发现

• 配备ToM组件的说话者模型在最终准确率与生成话语的流畅性方面，始终优于无ToM的基线模型。
• 在视觉与语义更相似的干扰项下训练，可导致生成话语更长、更精确、更流畅，即使性能提升并不总是显著。
• ToM听者模型使说话者能够生成更具语用意义的话语，选择那些最可能被听者正确理解的表达。
• 环境难度的提升促使语言发展更加复杂，表明环境压力推动了语言的精致化。
• 尽管有所改进，生成的语言在流畅性与自然度方面仍与人类水平的图像描述存在显著差距。
• ToM与环境压力的结合在计算语言习得中展现出巨大潜力，有助于开发更具语用意识与类人特征的语言模型。

Figure 2: An example image-caption pair, and distractors ranked as similar by various metrics. Caption and Hybrid similarities are computed with TF-IDF - weighted RoBERTa embeddings.