[Paper Review] Benchmarking Deep Learning Interpretability in Time Series Predictions

Aya Abdelsalam Ismail, Mohamed K. Gunady+2 more|arXiv (Cornell University)|Oct 26, 2020

Explainable Artificial Intelligence (XAI)40 references87 citations

TL;DR

The paper benchmarks saliency-based interpretability methods for multivariate time series across RNNs, TCNs, and Transformers, identifies systematic failures due to time-feature conflation, and proposes Temporal Saliency Rescaling (TSR) to improve explanations.

ABSTRACT

Saliency methods are used extensively to highlight the importance of input features in model predictions. These methods are mostly used in vision and language tasks, and their applications to time series data is relatively unexplored. In this paper, we set out to extensively compare the performance of various saliency-based interpretability methods across diverse neural architectures, including Recurrent Neural Network, Temporal Convolutional Networks, and Transformers in a new benchmark of synthetic time series data. We propose and report multiple metrics to empirically evaluate the performance of saliency methods for detecting feature importance over time using both precision (i.e., whether identified features contain meaningful signals) and recall (i.e., the number of features with signal identified as important). Through several experiments, we show that (i) in general, network architectures and saliency methods fail to reliably and accurately identify feature importance over time in time series data, (ii) this failure is mainly due to the conflation of time and feature domains, and (iii) the quality of saliency maps can be improved substantially by using our proposed two-step temporal saliency rescaling (TSR) approach that first calculates the importance of each time step before calculating the importance of each feature at a time step.

Motivation & Objective

• Assess how well common saliency methods identify informative features across time in multivariate time series.
• Evaluate the impact of neural architectures (RNNs, TCNs, Transformers) on saliency map quality.
• Develop a synthetic benchmark with known informative features to quantify precision and recall of explanations.
• Propose a practical method (TSR) to decouple time and feature relevance to improve saliency maps for time series.

Proposed method

• Compare gradient-based and perturbation-based saliency methods across LSTM, LSTM with Input-Cell Attention, TCN, and Transformer architectures.
• Design synthetic multivariate time series datasets with known informative features distributed over time and features.
• Evaluate using precision/recall metrics by progressively masking saliency-identified features and measuring accuracy drop.
• Propose Temporal Saliency Rescaling (TSR): first compute time-relevance by masking each time step, then compute feature-relevance within time steps above a time-threshold, final score is product of time and feature relevance.
• Provide an algorithmic description (Algorithm 1) of TSR and report its impact on saliency quality.
• Compare saliency maps on time-series data to those treated as images or univariate/time-augmented series.

Experimental results

Research questions

• RQ1 Do standard saliency methods reliably identify time-varying feature importance in multivariate time series across common neural architectures?
• RQ2 How do architecture choices (RNN, TCN, Transformer) influence the quality of saliency maps in time series?
• RQ3 Can a post-hoc temporal decoupling improve the fidelity of explanations without altering model predictions?
• RQ4 What is the trade-off between interpretability quality and computational cost when applying TSR?
• RQ5 Do saliency evaluations on time series differ when data are treated as images versus intrinsic time-series representations?

Key findings

• Most saliency methods fail to produce high-quality interpretations for multivariate time series, even when models achieve high accuracy.
• Saliency maps often highlight informative time steps but struggle to distinguish which features within those steps are truly informative.
• Model architecture significantly affects saliency quality, with differences sometimes larger than the choice of saliency method.
• A two-step Temporal Saliency Rescaling (TSR) approach substantially improves saliency quality across methods and architectures, though with higher computational cost.
• TSR yields the best performance in the benchmark metrics among simple baselines when applied to Gradient saliency, and demonstrates robustness across datasets.