[论文解读] Understanding Natural Language Understanding Systems. A Critical Analysis

Alessandro Lenci|arXiv (Cornell University)|Mar 1, 2023

Natural Language Processing Techniques被引用 8

一句话总结

本论文对近来自然语言理解系统进行了批判性分析，认为它们在具备类人语言学习能力的同时，在解释与推理方面存在显著差距，需要结构化知识整合来实现。

ABSTRACT

The development of machines that «talk like us», also known as Natural Language Understanding (NLU) systems, is the Holy Grail of Artificial Intelligence (AI), since language is the quintessence of human intelligence. The brief but intense life of NLU research in AI and Natural Language Processing (NLP) is full of ups and downs, with periods of high hopes that the Grail is finally within reach, typically followed by phases of equally deep despair and disillusion. But never has the trust that we can build «talking machines» been stronger than the one engendered by the last generation of NLU systems. But is it gold all that glitters in AI? do state-of-the-art systems possess something comparable to the human knowledge of language? Are we at the dawn of a new era, in which the Grail is finally closer to us? In fact, the latest achievements of AI systems have sparkled, or better renewed, an intense scientific debate on their true language understanding capabilities. Some defend the idea that, yes, we are on the right track, despite the limits that computational models still show. Others are instead radically skeptic and even dismissal: The present limits are not just contingent and temporary problems of NLU systems, but the sign of the intrinsic inadequacy of the epistemological and technological paradigm grounding them. This paper aims at contributing to such debate by carrying out a critical analysis of the linguistic abilities of the most recent NLU systems. I contend that they incorporate important aspects of the way language is learnt and processed by humans, but at the same time they lack key interpretive and inferential skills that it is unlikely they can attain unless they are integrated with structured knowledge and the ability to exploit it for language use.

研究动机与目标

• 评估最先进的自然语言理解系统是否展现出类人语言理解能力。
• 识别当前自然语言理解范式的认识论与技术限制。
• 评估当前系统能够捕捉到的语言能力以及缺失的部分。
• 讨论结构化知识在实现更深层语言使用中的作用。

提出的方法

• 对近期自然语言理解系统及其语言学习/处理能力进行批判性语言学分析。
• 将计算模型与文献中讨论的人类语言习得与处理模式进行比较。
• 论证整合结构化知识以实现解读与推理能力的必要性。

实验结果

研究问题

• RQ1 当前的自然语言理解系统在多大程度上具有类人水平的语言理解能力？
• RQ2 最前沿的自然语言理解系统缺失了哪些解释性或推理能力？
• RQ3 这些差距是由于当前计算范式的局限性，还是由于缺乏结构化知识整合？
• RQ4 整合结构化知识及其利用在多大程度上可以提升自然语言理解系统的语言使用？

主要发现

• 自然语言理解系统在语言学习与处理的某些重要方面与人类相似。
• 然而，它们缺乏关键的解释性与推理能力。
• 若要在语言理解上获得更大提升，可能需要与结构化知识及其利用机制相整合。