[Paper Review] Evaluating Explanation Methods for Neural Machine Translation

Jierui Li, Lemao Liu+4 more|arXiv (Cornell University)|May 4, 2020

Topic Modeling34 references4 citations

TL;DR

This paper proposes a fidelity-based metric to evaluate explanation methods for neural machine translation (NMT) by measuring how well the most relevant source words extracted by an explanation method can reconstruct the NMT model's predictive behavior. The method uses an approximate optimization approach to estimate fidelity without human annotations, and experiments show that prediction difference (Pd) outperforms attention, gradient norm, and weighted gradient methods in fidelity across six translation tasks.

ABSTRACT

Recently many efforts have been devoted to interpreting the black-box NMT models, but little progress has been made on metrics to evaluate explanation methods. Word Alignment Error Rate can be used as such a metric that matches human understanding, however, it can not measure explanation methods on those target words that are not aligned to any source word. This paper thereby makes an initial attempt to evaluate explanation methods from an alternative viewpoint. To this end, it proposes a principled metric based on fidelity in regard to the predictive behavior of the NMT model. As the exact computation for this metric is intractable, we employ an efficient approach as its approximation. On six standard translation tasks, we quantitatively evaluate several explanation methods in terms of the proposed metric and we reveal some valuable findings for these explanation methods in our experiments.

Motivation & Objective

• To address the lack of principled, automated metrics for evaluating explanation methods in neural machine translation.
• To evaluate explanation methods based on their fidelity to the NMT model's predictive behavior, rather than relying on human-annotated alignments.
• To develop a scalable, approximation-based method for estimating fidelity that works on all target words, including those not aligned in human annotations.
• To quantitatively compare multiple explanation methods—attention, gradient norm, weighted gradient, and prediction difference—on their ability to reflect the NMT model’s behavior.

Proposed method

• Proposes a principled evaluation metric based on fidelity: the degree to which a proxy model trained on the most relevant source words matches the NMT model’s predictions.
• Frames the metric as an optimization problem minimizing the expected disagreement between the NMT model and a proxy model built from selected source words.
• Approximates the intractable exact computation using a two-step machine learning strategy: empirical risk minimization followed by validation on a held-out test set.
• Constructs different proxy model architectures using the top-k relevant words identified by each explanation method to compute the approximate metric.
• Employs statistical learning theory to cast the optimization into a standard learning problem, enabling efficient and scalable evaluation.
• Uses perplexity as a proxy for model agreement, with lower perplexity indicating higher fidelity of the explanation method.

Experimental results

Research questions

• RQ1 How can explanation methods for NMT be evaluated based on their fidelity to the model’s predictive behavior rather than human-annotated alignments?
• RQ2 Can a metric be developed that evaluates explanation methods on all target words, including those not aligned to any source word?
• RQ3 How do different explanation methods—attention, gradient norm, weighted gradient, and prediction difference—compare in terms of fidelity to the NMT model’s behavior?
• RQ4 Is the proposed approximate metric effective and reliable for comparing explanation methods without human annotations?

Key findings

• The prediction difference (Pd) method achieves the highest fidelity among all evaluated explanation methods, outperforming attention, gradient norm, and weighted gradient.
• Gradient norm (Ngrad) and prediction difference (Pd) are both strong explanation methods, with Pd showing significantly better performance in reconstructing the NMT model’s predictions.
• The proposed fidelity-based metric successfully identifies high-quality explanation methods even on target words not covered by human word alignments, which AER cannot evaluate.
• The ranking of explanation methods by the proposed metric (SA) is consistent across three different translation tasks, unlike AER, which shows high task dependency.
• The evaluation results confirm that fidelity is a reliable and objective metric for comparing explanation methods, independent of human judgment or task-specific assumptions.