[Paper Review] Learning Important Features Through Propagating Activation Differences
DeepLIFT assigns input feature importances by backpropagating differences-from-reference through the network, revealing positive/negative contributions and addressing gradient saturation.
The purported "black box" nature of neural networks is a barrier to adoption in applications where interpretability is essential. Here we present DeepLIFT (Deep Learning Important FeaTures), a method for decomposing the output prediction of a neural network on a specific input by backpropagating the contributions of all neurons in the network to every feature of the input. DeepLIFT compares the activation of each neuron to its 'reference activation' and assigns contribution scores according to the difference. By optionally giving separate consideration to positive and negative contributions, DeepLIFT can also reveal dependencies which are missed by other approaches. Scores can be computed efficiently in a single backward pass. We apply DeepLIFT to models trained on MNIST and simulated genomic data, and show significant advantages over gradient-based methods. Video tutorial: http://goo.gl/qKb7pL, ICML slides: bit.ly/deeplifticmlslides, ICML talk: https://vimeo.com/238275076, code: http://goo.gl/RM8jvH.
Motivation & Objective
- Motivate interpretability for neural nets where transparency is critical.
- Introduce DeepLIFT as a difference-from-reference based importance scoring method.
- Address limitations of gradient-based approaches such as saturation and discontinuities.
- Propose rules to separate positive/negative contributions for richer explanations.
- Demonstrate efficiency and applicability to vision and genomics tasks.
Proposed method
- Define importance scores as contributions that sum to the difference-from-reference (Delta t).
- Introduce multipliers m_{Delta x Delta t} that propagate differences through the network (chain rule for multipliers).
- Specify reference activations y^0 by choosing a reference input and computing neuron activations from it.
- Allow separate tracking of Delta y^+ and Delta y^- to capture positive/negative contributions.
- Present Linear, Rescale, and RevealCancel rules to assign contributions at each neuron.
- Outline how to compute scores in a single backward pass similar to backpropagation.
Experimental results
Research questions
- RQ1Can importance scores be propagated and interpreted when gradients are zero or unstable due to saturation or discontinuities?
- RQ2Do difference-from-reference based scores (DeepLIFT) produce more faithful feature attributions than gradient-based methods in vision and genomics tasks?
- RQ3How do positive/negative contribution separations and RevealCancel improve attribution quality?
- RQ4What reference choices yield robust, interpretable explanations across domains?
Key findings
- DeepLIFT can propagate informative attributions even where gradients vanish, avoiding saturation artifacts.
- RevealCancel and separating positive/negative contributions improve alignment with meaningful motifs in synthetic genomic data and MNIST digit-confusion tasks.
- On MNIST, DeepLIFT with RevealCancel outperformed gradient-based methods in identifying pixels to flip digits between classes.
- In regulatory DNA simulations, DeepLIFT variants (especially with RevealCancel) highlighted true motif patterns more reliably than gradient-based methods.
- Integrated Gradients and gradient×input performed comparably on MNIST but lagged on motif discovery tasks, illustrating domain-dependent advantages.
- The method enables efficient, single-pass computation and provides a principled reference-based explanation framework.
Read this paper
Open the full text in the Nubint reader and ask about any passage as you read.
Read this paper on NubintThis review was generated by AI from the original paper. Please check the paper before citing it.