[论文解读] LMExplainer: Grounding Knowledge and Explaining Language Models

Zichen Chen, Ambuj K. Singh+4 more|arXiv (Cornell University)|Mar 29, 2023

Explainable Artificial Intelligence (XAI)被引用 7

一句话总结

LMExplainer 使用知识图谱和图神经网络提取推理要素并生成自然语言解释，以LM预测为基础，在 CommonsenseQA 上达到最先进的结果并在 OpenBookQA 上具备竞争力，同时提供可解释的为何选与为何不选的解释。

ABSTRACT

Language models (LMs) like GPT-4 are important in AI applications, but their opaque decision-making process reduces user trust, especially in safety-critical areas. We introduce LMExplainer, a novel knowledge-grounded explainer that clarifies the reasoning process of LMs through intuitive, human-understandable explanations. By leveraging a graph attention network (GAT) with a large-scale knowledge graph (KG), LMExplainer not only precisely narrows the reasoning space to focus on the most relevant knowledge but also grounds its reasoning in structured, verifiable knowledge to reduce hallucinations and enhance interpretability. LMExplainer effectively generates human-understandable explanations to enhance transparency and streamline the decision-making process. Additionally, by incorporating debugging into the explanation, it offers expertise suggestions that improve LMs from a developmental perspective. Thus, LMExplainer stands as an enhancement in making LMs more accessible and understandable to users. We evaluate LMExplainer on benchmark datasets such as CommonsenseQA and OpenBookQA, demonstrating that it outperforms most existing methods. By comparing the explanations generated by LMExplainer with those of other models, we show that our approach offers more comprehensive and clearer explanations of the reasoning process. LMExplainer provides a deeper understanding of the inner workings of LMs, advancing towards more reliable, transparent, and equitable AI.

研究动机与目标

• 使大语言模型（LLMs）在超越基于注意力的方法的前提下具有可解释的解释需求的动机。
• 提出 LMExplainer，在知识图谱和图神经网络框架中对 LM 推理进行锚定。
• 实现生成人类可理解的解释，反映模型的推理过程。

提出的方法

• 从输入和检索的KG中提取关键信息，构建元素图 G_e。
• 在 G_e 上使用图注意力网络（GAT）以识别支撑预测的推理要素。
• 计算注意力驱动的影响信号，以引导预测并生成解释。
• 通过一个基于指令的生成器（GPT-3.5-turbo）利用提取的组件生成为何选与为何不选的解释。
• 使用与 LM 与解释嵌入的交叉熵损失进行训练，以预测正确答案。

Figure 1: The LMExplainer architecture. Given a question context $z$ (which includes question $q$ and the set $\mathcal{A}$ of answers), we first generate language embeddings using a pre-trained LM. Simultaneously, it retrieves relevant knowledge from a KG to construct a subgraph (Section 4.1 ). The

实验结果

研究问题

• RQ1 我们如何为语言模型的决策过程提供可解释且自然的解释？
• RQ2 提供解释是否会影响语言模型在问答任务上的性能？
• RQ3 基于知识的图方法是否能为 LM 预测提供可信且有用的解释？

主要发现

Method	IHdev-Acc.	IHtest-Acc.
MHGRN (2020)	73.69%	71.08%
KagNet (2019)	73.47%	69.01%
GconAttn (2019)	72.61%	68.59%
RGCN (2018)	72.69%	68.41%
RN (2017)	74.57%	69.08%
GreaseLM (2022)	76.17%	72.60%
QA-GNN (2021)	74.94%	72.36%
LMExplainer (ours)	77.97%	77.31%

• LMExplainer 在 CommonsenseQA 上超越了最先进的 LM+KG 方法（IHtest 77.31% 对比 71.08% 的最佳非 LM-Explainer 基线）。
• LMExplainer 在 OpenBookQA 上实现了具竞争力的性能（测试准确率 68.00%，对比 GreaseLM 的 70.80%）。
• 在消融实验中，结合外部知识与解释组件显著提升准确性，解释组件对性能有强烈影响。
• 解释包含排序的推理要素，以及为何选和为何不选的叙述，为模型推理提供人类可理解的洞察。
• 相比 PathReasoner 与 ECQA，LMExplainer 提供完整的推理路径及所选答案的正当性。