[Paper Review] A Self-Training Method for Machine Reading Comprehension with Soft Evidence Extraction

Yilin Niu, Fangkai Jiao+4 more|arXiv (Cornell University)|May 11, 2020

Topic Modeling42 references4 citations

TL;DR

This paper proposes a self-training method (STM) for machine reading comprehension that improves evidence extraction without golden labels by iteratively generating and refining soft evidence labels using a base model's predictions. The method boosts performance across seven datasets in extractive and non-extractive MRC tasks, with significant gains in evidence extraction accuracy—up to +40% on CoQA—demonstrating robustness and generalization across diverse MRC tasks and base models like BERT and RoBERTa.

ABSTRACT

Neural models have achieved great success on machine reading comprehension (MRC), many of which typically consist of two components: an evidence extractor and an answer predictor. The former seeks the most relevant information from a reference text, while the latter is to locate or generate answers from the extracted evidence. Despite the importance of evidence labels for training the evidence extractor, they are not cheaply accessible, particularly in many non-extractive MRC tasks such as YES/NO question answering and multi-choice MRC. To address this problem, we present a Self-Training method (STM), which supervises the evidence extractor with auto-generated evidence labels in an iterative process. At each iteration, a base MRC model is trained with golden answers and noisy evidence labels. The trained model will predict pseudo evidence labels as extra supervision in the next iteration. We evaluate STM on seven datasets over three MRC tasks. Experimental results demonstrate the improvement on existing MRC models, and we also analyze how and why such a self-training method works in MRC. The source code can be obtained from https://github.com/SparkJiao/Self-Training-MRC

Motivation & Objective

• To address the challenge of limited or unavailable golden evidence labels in non-extractive MRC tasks such as YES/NO QA and multi-choice MRC.
• To develop a general, label-efficient method that improves evidence extraction without relying on external resources or manual annotation.
• To enable iterative refinement of evidence predictions through self-training, where pseudo-labels from prior iterations supervise subsequent model updates.
• To evaluate the method’s effectiveness and stability across diverse MRC tasks and base models, including BERT and RoBERTa.
• To analyze error propagation and self-correction dynamics in the self-training loop to ensure robustness.

Proposed method

• The method employs an iterative self-training loop: at each iteration, a base MRC model is trained on golden answers and noisy pseudo-evidence labels from the previous iteration.
• Pseudo-evidence labels are generated by the model itself after each training step, forming a feedback loop that improves evidence extraction over time.
• The evidence extractor component is supervised using these auto-generated labels, enabling end-to-end training without human-annotated evidence.
• The approach is general and applicable to any MRC model architecture, including BERT and DSQA, and works across extractive and non-extractive tasks.
• Soft evidence extraction uses differentiable attention or span prediction to produce continuous or probabilistic evidence scores, allowing gradient-based optimization.
• The method avoids reinforcement learning and complex label refinement techniques, relying instead on stable, iterative refinement of pseudo-labels.

Experimental results

Research questions

• RQ1 Can self-training with automatically generated pseudo-evidence labels improve evidence extraction in MRC when golden labels are unavailable?
• RQ2 How does the iterative refinement of pseudo-labels affect the performance of the evidence extractor and the final answer predictor?
• RQ3 To what extent does error propagation occur in the self-training loop, and can the method self-correct from incorrect predictions?
• RQ4 Does the method generalize across different MRC tasks, including YES/NO QA, multi-choice MRC, and open-domain QA?
• RQ5 How does the performance of STM vary across different base models, especially weaker ones like BERT-base versus stronger ones like RoBERTa-large?

Key findings

• STM significantly improved evidence extraction accuracy by up to 40% on the CoQA development set when applied to RoBERTa-HA, from 13.8% to 19.3%.
• BERT-HA+STM outperformed the base BERT-HA model across all metrics on CoQA and MultiRC, demonstrating consistent gains in precision and recall.
• The proportion of incorrect evidence predictions decreased from 80% in the initial iteration (BERT-HA) to 60% after three self-training iterations, with 27% of wrong predictions corrected with high confidence.
• Only 4% of predictions were mistakenly revised and propagated, but nearly 50% of such errors were corrected in later iterations, showing resilience to error propagation.
• The method achieved consistent improvements across seven datasets spanning three MRC tasks, proving its generalization and effectiveness.
• Despite strong base models like RoBERTa-HA achieving high answer accuracy (92.6%), STM still improved evidence extraction significantly, indicating that better evidence helps even strong models.