[论文解读] Machine Reading Comprehension: The Role of Contextualized Language Models and Beyond

Zhuosheng Zhang, Hai Zhao+1 more|arXiv (Cornell University)|May 13, 2020

Topic Modeling参考文献 210被引用 48

一句话总结

One or two sentence direct-answer summary

ABSTRACT

Machine reading comprehension (MRC) aims to teach machines to read and comprehend human languages, which is a long-standing goal of natural language processing (NLP). With the burst of deep neural networks and the evolution of contextualized language models (CLMs), the research of MRC has experienced two significant breakthroughs. MRC and CLM, as a phenomenon, have a great impact on the NLP community. In this survey, we provide a comprehensive and comparative review on MRC covering overall research topics about 1) the origin and development of MRC and CLM, with a particular focus on the role of CLMs; 2) the impact of MRC and CLM to the NLP community; 3) the definition, datasets, and evaluation of MRC; 4) general MRC architecture and technical methods in the view of two-stage Encoder-Decoder solving architecture from the insights of the cognitive process of humans; 5) previous highlights, emerging topics, and our empirical analysis, among which we especially focus on what works in different periods of MRC researches. We propose a full-view categorization and new taxonomies on these topics. The primary views we have arrived at are that 1) MRC boosts the progress from language processing to understanding; 2) the rapid improvement of MRC systems greatly benefits from the development of CLMs; 3) the theme of MRC is gradually moving from shallow text matching to cognitive reasoning.

研究动机与目标

• Explain the origin and development of MRC and the pivotal role of CLMs.
• Survey how CLMs impact MRC performance and NLP communities.
• Define MRC scope, datasets, evaluation, and typical architectures.
• Propose a two-stage Encoder-Decoder view of MRC inspired by cognitive processes.
• Highlight emerging topics, Herausforderungen, and future opportunities in MRC.

提出的方法

• Provide a full-view taxonomy of MRC topics and CLM interactions.
• Compare CLM derivatives and their training objectives (MLM, PLM, AE) and architectures (RNN, Transformer, Transformer-XL).
• Analyze how MRC formation translates traditional NLP tasks into QA-like or span-based formats.
• Discuss empirical findings and trends across flagship MRC datasets and leaderboards.
• Offer insights on interpretability, reasoning, and resource-efficient model design.

实验结果

研究问题

• RQ1 How have CLMs influenced the performance and capabilities of MRC over time?
• RQ2 What architectural and training-objective choices drive gains in MRC, and how do they relate to cognitive-inspired solving strategies?
• RQ3 In what ways can traditional NLP tasks be reformulated as MRC, and what are the implications for NLP research?
• RQ4 What are the emerging topics and future opportunities for advancing MRC beyond current QA-focused benchmarks?

主要发现

• MRC advances are closely tied to the development of CLMs, enabling richer sentence-level representations.
• Two-stage Encoder-Decoder perspectives informed by cognitive processes help categorize MRC architectures and methods.
• CLMs have driven rapid performance gains on MRC benchmarks, while raising questions about genuine understanding versus pattern matching.
• MRC serves as a valuable testbed for language representations and multi-task transfer to other NLP tasks.
• Future directions include interpretable datasets, complex reasoning, low-resource MRC, and multimodal grounding.