[Paper Review] TSInsight: A local-global attribution framework for interpretability in time-series data

Shoaib Ahmed Siddiqui, Dominique Mercier+2 more|arXiv (Cornell University)|Apr 6, 2020

Explainable Artificial Intelligence (XAI)27 references4 citations

TL;DR

TSInsight proposes a novel local-global attribution framework for time-series interpretability by integrating a sparsity-regularized auto-encoder with a deep learning classifier. It jointly optimizes for prediction fidelity and reconstruction of salient features, enabling both instance-specific and model-wide explanations while achieving input space contraction and superior interpretability across 8 datasets.

ABSTRACT

With the rise in the employment of deep learning methods in safety-critical scenarios, interpretability is more essential than ever before. Although many different directions regarding interpretability have been explored for visual modalities, time-series data has been neglected with only a handful of methods tested due to their poor intelligibility. We approach the problem of interpretability in a novel way by proposing TSInsight where we attach an auto-encoder to the classifier with a sparsity-inducing norm on its output and fine-tune it based on the gradients from the classifier and a reconstruction penalty. TSInsight learns to preserve features that are important for prediction by the classifier and suppresses those that are irrelevant i.e. serves as a feature attribution method to boost interpretability. In contrast to most other attribution frameworks, TSInsight is capable of generating both instance-based and model-based explanations. We evaluated TSInsight along with 9 other commonly used attribution methods on 8 different time-series datasets to validate its efficacy. Evaluation results show that TSInsight naturally achieves output space contraction, therefore, is an effective tool for the interpretability of deep time-series models.

Motivation & Objective

• To address the lack of interpretable methods for deep learning in time-series data, especially in safety-critical domains like healthcare and autonomous systems.
• To develop a unified framework that provides both instance-based and model-based explanations, overcoming the limitations of existing attribution methods focused solely on individual predictions.
• To improve interpretability by inducing input space contraction through a sparsity-inducing norm on the auto-encoder's output, enhancing feature relevance and intelligibility.
• To evaluate the method across diverse time-series datasets and compare it comprehensively with 9 baseline attribution techniques.
• To demonstrate that the method achieves out-of-the-box properties such as generic applicability and robustness to input perturbations.

Proposed method

• A deep learning classifier is coupled with a variational auto-encoder (VAE) that is jointly fine-tuned using gradients from the classifier and a reconstruction loss.
• A sparsity-inducing norm (e.g., L1) is applied to the auto-encoder’s output to encourage the model to reconstruct only features relevant to the classifier’s prediction.
• The optimization process preserves features that are causally or correlated with the prediction, suppressing irrelevant or noisy components.
• The method enables dual explanation modes: instance-based (per-input) and model-based (dataset-level) by solving the optimization over individual inputs or the full dataset.
• The auto-encoder’s Jacobian spectrum is analyzed to verify input space contraction, indicating that the model learns to compress irrelevant variations.
• The framework is compatible with various backbone architectures, including CNNs and LSTMs, demonstrating architectural generality.

Experimental results

Research questions

• RQ1 Can a unified framework generate both instance-specific and model-wide explanations for time-series deep learning models?
• RQ2 Does incorporating a sparsity-inducing norm on the auto-encoder output improve feature attribution and interpretability in time-series data?
• RQ3 To what extent does TSInsight achieve input space contraction, and how does this relate to improved model interpretability?
• RQ4 How does TSInsight compare to 9 established attribution methods in terms of saliency estimation and explanation plausibility across diverse time-series datasets?
• RQ5 What are the out-of-the-box properties of TSInsight, such as generic applicability and robustness, across different model architectures?

Key findings

• TSInsight achieved the most plausible and competitive saliency estimations across all 8 time-series datasets, outperforming 9 baseline methods in explanation quality.
• The method induced significant input space contraction, as evidenced by the sharp decay in the singular values of the auto-encoder’s average Jacobian, indicating suppression of non-informative directions.
• TSInsight demonstrated generic applicability by successfully extracting salient features from both CNN and LSTM architectures, with saliency patterns aligning with architectural behavior (e.g., peak saliency at the last time step in LSTMs).
• The suppression test showed that TSInsight effectively reduced prediction confidence when suppressing top features, confirming its sensitivity to relevant input components.
• The framework naturally produces model-based explanations by optimizing over the full dataset, enabling global insights into feature importance across the data distribution.
• TSInsight achieved slight improvements in adversarial robustness due to input space contraction, though gains were inconsistent across datasets and not uniformly strong.