[Paper Review] Explaining Question Answering Models through Text Generation

Veronica Latcinnik, Jonathan Berant|arXiv (Cornell University)|Apr 12, 2020

Topic ModelingComputer Science53 references44 citations

TL;DR

The paper presents a generator–classifier QA model where a language-model-based generator outputs a textual hypothesis used by a classifier to select the answer, enabling interpretability of knowledge used by the LM without sacrificing competitive performance.

ABSTRACT

Large pre-trained language models (LMs) have been shown to perform surprisingly well when fine-tuned on tasks that require commonsense and world knowledge. However, in end-to-end architectures, it is difficult to explain what is the knowledge in the LM that allows it to make a correct prediction. In this work, we propose a model for multi-choice question answering, where a LM-based generator generates a textual hypothesis that is later used by a classifier to answer the question. The hypothesis provides a window into the information used by the fine-tuned LM that can be inspected by humans. A key challenge in this setup is how to constrain the model to generate hypotheses that are meaningful to humans. We tackle this by (a) joint training with a simple similarity classifier that encourages meaningful hypotheses, and (b) by adding loss functions that encourage natural text without repetitions. We show on several tasks that our model reaches performance that is comparable to end-to-end architectures, while producing hypotheses that elucidate the knowledge used by the LM for answering the question.

Motivation & Objective

• Investigate explainable LM-based models for multi-choice QA to reveal the knowledge used for answering.
• Develop a generator that outputs human-interpretable textual hypotheses conditioned on questions.
• Train a classifier that relies on generated hypotheses, and constrain hypotheses to be meaningful and natural.

Proposed method

• Use a pre-trained autoregressive LM (GPT-2 or XLNet) as the generator to produce a textual hypothesis from a question.
• Couple the generator with a classifier that predicts the correct answer using the question, hypothesis, and answer options.
• Address non-differentiability of discrete generation with straight-through Gumbel-Softmax for end-to-end training.
• Incorporate a similarity classifier (word-embedding based) to encourage meaningful hypotheses that must be used by the downstream model.
• Jointly train a stronger LM-based classifier with the similarity classifier to ensure the hypothesis informs the decision.
• Apply additional explainability mechanisms such as KL-divergence regularization, repetition penalties, and Top-K decoding to improve hypothesis quality and variety.

Experimental results

Research questions

• RQ1 What knowledge encoded in a language model is actually used to answer a question in a QA task?
• RQ2 Can we generate human-interpretable textual hypotheses that reflect the LM’s internal knowledge and still achieve competitive QA performance?
• RQ3 How do training objectives and decoding strategies affect the usefulness and interpretability of the generated hypotheses?
• RQ4 Do joint training and auxiliary losses prevent the classifier from ignoring the generated hypotheses?
• RQ5 Can the approach generalize to other QA domains or zero-shot transfers?

Key findings

• The proposed generator–classifier setup achieves QA performance comparable to end-to-end models on CSQA and shows transfer capability to QASC in zero-shot settings.
• Generated hypotheses provide a window into the LM’s knowledge; they can reveal missing knowledge, semantic errors, and how hypotheses influence predictions.
• A simple similarity classifier encourages meaningful hypotheses and helps ensure the LM-based classifier utilizes the hypothesis for prediction.
• Joint training with an LM-based classifier preserves reliance on the hypothesis, whereas relying solely on the similarity classifier can lead to predictions that ignore the hypothesis.
• Top-K ST decoding often yields the best QA accuracy among weakly supervised generators, while also providing diverse, human-interpretable hypotheses; excessive objective pressures can reduce accuracy but improve naturalness.
• Human evaluation indicates a trade-off: hypotheses optimized for QA are less “natural,” while more natural hypotheses may slightly hurt accuracy but improve interpretability.