[Paper Review] Open Domain Question Answering over Virtual Documents: A Unified Approach for Data and Text.

Kaixin Ma, Hao Cheng+3 more|arXiv (Cornell University)|Oct 16, 2021

Topic Modeling43 references4 citations

TL;DR

This paper proposes UDT-QA, a unified framework for open-domain question answering that integrates verbalized Wikipedia tables and Wikidata triples as augmented knowledge sources. By leveraging data-to-text generation to enhance retrieval and reasoning, UDT-QA achieves state-of-the-art performance on Natural Questions, demonstrating significant gains over text-only baselines, especially in reasoning over structured knowledge.

ABSTRACT

Due to its potential for a universal interface over both data and text, data-to-text generation is becoming increasingly popular recently. However, few previous work has focused on its application to downstream tasks, e.g. using the converted data for grounding or reasoning. In this work, we aim to bridge this gap and use the data-to-text method as a means for encoding structured knowledge for knowledge-intensive applications, i.e. open-domain question answering (QA). Specifically, we propose a verbalizer-retriever-reader framework for open-domain QA over data and text where verbalized tables from Wikipedia and triples from Wikidata are used as augmented knowledge sources. We show that our Unified Data and Text QA, UDT-QA, can effectively benefit from the expanded knowledge index, leading to large gains over text-only baselines. Notably, our approach sets the single-model state-of-the-art on Natural Questions. Furthermore, our analyses indicate that verbalized knowledge is preferred for answer reasoning for both adapted and hot-swap settings.

Motivation & Objective

• Address the gap in leveraging data-to-text generation for downstream knowledge-intensive tasks like open-domain question answering.
• Integrate structured knowledge from Wikipedia tables and Wikidata triples into a unified QA framework to improve reasoning and grounding.
• Develop a verbalizer-retriever-reader architecture that treats both data and text as unified knowledge sources for improved QA performance.
• Evaluate the effectiveness of verbalized knowledge in both adapted and hot-swap settings for reasoning and retrieval.
• Establish a new single-model state-of-the-art on the Natural Questions benchmark using unified data and text knowledge.

Proposed method

• Use a verbalizer module to convert structured knowledge from Wikipedia tables and Wikidata triples into natural language text for indexing.
• Employ a retriever component that retrieves relevant verbalized knowledge passages from a unified index of text and verbalized data.
• Apply a reader model to generate answers based on the retrieved passages, enabling end-to-end reasoning over both text and structured data.
• Construct a unified knowledge index that combines raw text with data-to-text generated descriptions of tables and triples.
• Design a framework that supports both adapted and hot-swap settings, enabling flexible integration of verbalized knowledge during inference.
• Utilize retrieval-augmented generation techniques to enhance answer generation by grounding in both textual and structured knowledge sources.

Experimental results

Research questions

• RQ1 Can data-to-text generated knowledge improve open-domain question answering when integrated with textual knowledge?
• RQ2 How does the inclusion of verbalized tables and Wikidata triples affect retrieval and answer generation performance compared to text-only baselines?
• RQ3 Does verbalized knowledge provide consistent benefits across different QA settings, including adapted and hot-swap configurations?
• RQ4 To what extent does the unified data and text knowledge index improve reasoning over structured knowledge in open-domain QA?
• RQ5 Can a single-model approach achieve state-of-the-art performance on benchmark QA datasets like Natural Questions using this unified knowledge encoding?

Key findings

• UDT-QA achieves a new single-model state-of-the-art on the Natural Questions benchmark by integrating verbalized Wikipedia tables and Wikidata triples.
• The model demonstrates large performance gains over text-only baselines, indicating that expanded knowledge indexing significantly improves QA accuracy.
• Verbalized knowledge is consistently preferred for answer reasoning in both adapted and hot-swap settings, suggesting robustness and generalization across configurations.
• The unified index of data and text enables more effective retrieval and grounding, particularly for complex reasoning over structured facts.
• The verbalizer-retriever-reader framework effectively leverages both modalities, showing that data-to-text generation enhances downstream QA when used as a knowledge encoding strategy.
• The results confirm that structured knowledge, when properly verbalized and indexed, contributes meaningfully to answer generation and reasoning in open-domain QA.