[Paper Review] Learning from the Best: Rationalizing Prediction by Adversarial Information Calibration

Lei Sha, Oana-Maria Camburu+1 more|arXiv (Cornell University)|Dec 16, 2020

Topic Modeling43 references4 citations

TL;DR

This paper proposes a novel adversarial information calibration method for extractive rationale extraction in neural models, where a selector-predictor model learns to mimic a black-box 'guider' model by aligning their internal representations. It further introduces a language model-based regularizer to ensure fluent, coherent rationales, significantly improving rationale precision and recall without sacrificing prediction accuracy on sentiment and legal judgment tasks.

ABSTRACT

Explaining the predictions of AI models is paramount in safety-critical applications, such as in legal or medical domains. One form of explanation for a prediction is an extractive rationale, i.e., a subset of features of an instance that lead the model to give its prediction on the instance. Previous works on generating extractive rationales usually employ a two-phase model: a selector that selects the most important features (i.e., the rationale) followed by a predictor that makes the prediction based exclusively on the selected features. One disadvantage of these works is that the main signal for learning to select features comes from the comparison of the answers given by the predictor and the ground-truth answers. In this work, we propose to squeeze more information from the predictor via an information calibration method. More precisely, we train two models jointly: one is a typical neural model that solves the task at hand in an accurate but black-box manner, and the other is a selector-predictor model that additionally produces a rationale for its prediction. The first model is used as a guide to the second model. We use an adversarial-based technique to calibrate the information extracted by the two models such that the difference between them is an indicator of the missed or over-selected features. In addition, for natural language tasks, we propose to use a language-model-based regularizer to encourage the extraction of fluent rationales. Experimental results on a sentiment analysis task as well as on three tasks from the legal domain show the effectiveness of our approach to rationale extraction.

Motivation & Objective

• To improve the reliability and interpretability of black-box neural models in safety-critical domains like medicine and law.
• To address the limitation of prior selector-predictor models that rely solely on prediction accuracy as a learning signal for rationale selection.
• To reduce the exploration space for optimal rationale selection by leveraging information from a high-performing 'guider' model.
• To ensure extracted rationales are semantically fluent and coherent, avoiding arbitrary token selections due to adjacency.
• To develop a method that jointly optimizes rationale quality and prediction performance through adversarial calibration and language modeling.

Proposed method

• Train a 'guider' model as a high-accuracy black-box reference for the task, which is not required to be interpretable.
• Train a selector-predictor model that selects features and makes predictions based only on them, with a separate rationale output.
• Use an adversarial information calibration technique to align the final representation vectors of the guider and selector-predictor models, minimizing their divergence.
• Apply an information bottleneck technique to both models to ensure that only the minimal necessary information is retained, promoting concise rationales.
• Introduce a language model-based regularizer that encourages the selected tokens to form fluent, consecutive subphrases by modeling the likelihood of rationale sequences.
• Optimize the entire system end-to-end using gradient-based training, with the discriminator distinguishing between representations from the two models.

Experimental results

Research questions

• RQ1 Can adversarial calibration between a black-box guider model and a selector-predictor model improve rationale quality beyond prediction accuracy alone?
• RQ2 How does incorporating a language model regularizer affect the fluency and coherence of extracted rationales in NLP tasks?
• RQ3 To what extent does the proposed method improve precision and recall of rationale extraction compared to state-of-the-art baselines?
• RQ4 Can the method generalize across different domains, such as sentiment analysis and legal judgment prediction, while maintaining high prediction accuracy?
• RQ5 Does the information bottleneck component effectively reduce overfitting and redundant feature selection in the rationale?

Key findings

• The proposed method significantly improves rationale extraction performance, achieving higher precision and recall than previous state-of-the-art models on both sentiment analysis and legal judgment tasks.
• The language model-based regularizer effectively promotes fluent and coherent rationales by discouraging the selection of isolated or non-sequential tokens.
• Adversarial information calibration enables the selector-predictor model to learn more accurate and relevant feature selections by leveraging the high-quality representations of the guider model.
• The method maintains or even improves prediction accuracy while simultaneously enhancing interpretability, demonstrating a strong trade-off mitigation between performance and explainability.
• Human evaluations on legal judgment tasks confirmed that rationales generated by the proposed method are more semantically meaningful and closer to human-annotated rationales than those from baseline models.
• The ablation study showed that both the adversarial calibration and the language model regularizer contribute significantly to the overall performance gain.