[Paper Review] One Question Answering Model for Many Languages with Cross-lingual Dense Passage Retrieval

Akari Asai, Xinyan Yu+2 more|arXiv (Cornell University)|Jul 26, 2021

Topic Modeling31 citations

TL;DR

CORA is a unified many-to-many multilingual open QA model that retrieves evidence across languages with a single cross-lingual retriever and generates answers in the target language without translation or language-specific modules. It uses iterative data mining to expand training data and achieves state-of-the-art results across 26 languages.

ABSTRACT

We present Cross-lingual Open-Retrieval Answer Generation (CORA), the first unified many-to-many question answering (QA) model that can answer questions across many languages, even for ones without language-specific annotated data or knowledge sources. We introduce a new dense passage retrieval algorithm that is trained to retrieve documents across languages for a question. Combined with a multilingual autoregressive generation model, CORA answers directly in the target language without any translation or in-language retrieval modules as used in prior work. We propose an iterative training method that automatically extends annotated data available only in high-resource languages to low-resource ones. Our results show that CORA substantially outperforms the previous state of the art on multilingual open QA benchmarks across 26 languages, 9 of which are unseen during training. Our analyses show the significance of cross-lingual retrieval and generation in many languages, particularly under low-resource settings.

Motivation & Objective

• Address multilingual open QA without language-specific pipelines or translation-based error propagation.
• Develop a single cross-lingual retriever and a multilingual generator for answering questions in any target language from multilingual passages.
• Automatically expand training data across languages via iterative training and Wikipedia language links.
• Evaluate in diverse languages to demonstrate benefits of cross-lingual retrieval and generation, especially in low-resource settings.

Proposed method

• Extend Dense Passage Retrieval to a multilingual setting (mDPR) to encode passages and questions with mBERT and retrieve top passages via inner product.
• Use a multilingual sequence-to-sequence generator (mGEN, e.g., mT5) to produce answers in the target language conditioned on retrieved passages.
• Train via iterative data mining that expands training data using Wikipedia language links and model predictions, without extra human annotations or MT.
• Append a language tag to questions to indicate target language during generation.
• Translate-only data augmentation is avoided; synthetic answers are produced by the generator and used for training.
• In inference, CORA retrieves from any language and generates in the target language using a single mDPR/mGEN pair.

Experimental results

Research questions

• RQ1 Can a single cross-lingual retriever and generator enable open QA across many languages without language-specific pipelines?
• RQ2 How does cross-lingual retrieval impact answer quality in low-resource languages?
• RQ3 To what extent can iterative data mining extend training data across languages and improve cross-lingual QA performance?
• RQ4 Does multilingual generation improve answers compared to translation-based or extractive approaches?

Key findings

• CORA substantially outperforms prior multilingual open QA methods across 26 languages, with notable gains in Xor-TyDi QA and MKQA benchmarks.
• In Xor-TyDi QA, CORA achieves up to 23.4 F1-point improvement over the previous SOTA.
• In MKQA, CORA improves by up to 4.7 F1 points, even when MKQA data is not used for training.
• CORA sees benefits from cross-lingual retrieval and generation, especially in low-resource settings and for languages unseen during training.
• Retrieval from non-English sources enables answering a portion of questions otherwise deemed unanswerable due to evidence gaps in English sources.