[Paper Review] Retrieving and Reading: A Comprehensive Survey on Open-domain Question Answering

Fengbin Zhu, Wenqiang Lei+4 more|arXiv (Cornell University)|Jan 4, 2021

Topic Modeling160 references153 citations

TL;DR

This survey analyzes Open-domain QA with a focus on Retriever-Reader architectures, surveys retrieval methods (sparse, dense, iterative), and discusses neural MRC, challenges, and benchmarks.

ABSTRACT

Open-domain Question Answering (OpenQA) is an important task in Natural Language Processing (NLP), which aims to answer a question in the form of natural language based on large-scale unstructured documents. Recently, there has been a surge in the amount of research literature on OpenQA, particularly on techniques that integrate with neural Machine Reading Comprehension (MRC). While these research works have advanced performance to new heights on benchmark datasets, they have been rarely covered in existing surveys on QA systems. In this work, we review the latest research trends in OpenQA, with particular attention to systems that incorporate neural MRC techniques. Specifically, we begin with revisiting the origin and development of OpenQA systems. We then introduce modern OpenQA architecture named "Retriever-Reader" and analyze the various systems that follow this architecture as well as the specific techniques adopted in each of the components. We then discuss key challenges to developing OpenQA systems and offer an analysis of benchmarks that are commonly used. We hope our work would enable researchers to be informed of the recent advancement and also the open challenges in OpenQA research, so as to stimulate further progress in this field.

Motivation & Objective

• Trace the origin and development of OpenQA systems from traditional to neural approaches.
• Introduce and analyze the Retriever-Reader architecture and its components.
• Survey sparse, dense, and iterative retrievers and their roles in OpenQA.
• Discuss key challenges in OpenQA and provide an overview of commonly used benchmarks.

Proposed method

• Review the evolution of OpenQA from traditional pipeline to modern neural end-to-end systems.
• Propose a taxonomy of Retriever-Reader OpenQA systems and analyze component techniques.
• Classify retrievers into Sparse, Dense, and Iterative, and describe their mechanisms and trade-offs.
• Discuss end-to-end training paradigms and neural MRC models used for answer extraction.
• Summarize challenges and benchmarks to guide future research in OpenQA.

Experimental results

Research questions

• RQ1 What is the historical development of Open-domain QA and how have neural MRC methods shaped modern systems?
• RQ2 How does the Retriever-Reader architecture work, and what are the main variants and techniques for each component?
• RQ3 What are the relative strengths and limitations of Sparse, Dense, and Iterative retrievers in OpenQA?
• RQ4 What are the key challenges and benchmark datasets commonly used to evaluate OpenQA systems?

Key findings

• OpenQA systems are commonly organized into textual QA and KB-QA, with OpenQA aiming to answer questions from unstructured text.
• The modern prevalent architecture is Retriever-Reader, often augmented by document/answer post-processing and end-to-end training.
• Retrievers are categorized into Sparse, Dense, and Iterative, each with distinct mechanisms and trade-offs for document retrieval.
• Neural MRC models have become central to Answer Extraction, enabling end-to-end training and integration with retrievers.
• Dense retrievers address term mismatches via latent representations, while iterative retrievers enable multi-hop retrieval for complex questions.