[Paper Review] Time Interpret: a Unified Model Interpretability Library for Time Series

Joseph Enguehard|arXiv (Cornell University)|Jun 5, 2023

Explainable Artificial Intelligence (XAI)4 citations

TL;DR

Time Interpret introduces a unified PyTorch-based library for explainable AI in time series, extending Captum with specialized attribution methods like Temporal Integrated Gradients and Time Forward Tunnel. It enables model-agnostic, temporal feature attribution with support for real and synthetic datasets, evaluation metrics, and PyTorch Lightning integration, enhancing interpretability for high-stakes applications in finance and healthcare.

ABSTRACT

We introduce $ exttt{time_interpret}$, a library designed as an extension of Captum, with a specific focus on temporal data. As such, this library implements several feature attribution methods that can be used to explain predictions made by any Pytorch model. $ exttt{time_interpret}$ also provides several synthetic and real world time series datasets, various PyTorch models, as well as a set of methods to evaluate feature attributions. Moreover, while being primarily developed to explain predictions based on temporal data, some of its components have a different application, including for instance methods explaining predictions made by language models. In this paper, we give a general introduction of this library. We also present several previously unpublished feature attribution methods, which have been developed along with $ exttt{time_interpret}$.

Motivation & Objective

• Address the lack of unified, accessible tools for interpreting deep learning models on time series data, especially in high-stakes domains like finance and medicine.
• Extend Captum with time-series-specific attribution methods to improve model transparency and fairness in sequential decision-making.
• Provide a comprehensive toolkit including datasets, models, evaluation metrics, and PyTorch Lightning integration to streamline reproducible research and deployment.
• Support both temporal attribution (per-time-step feature importance) and model-agnostic explanation techniques applicable beyond time series, including NLP tasks.
• Introduce novel attribution methods such as TIG, Time Forward Tunnel, and LOF-enhanced LIME/SKAP to improve robustness and fidelity in explanation.

Proposed method

• Implement Temporal Integrated Gradients (TIG), a variant of Integrated Gradients that interpolates only the current time step while preserving past inputs and excluding future data.
• Introduce Time Forward Tunnel, a wrapper that applies any Captum attribution method iteratively on truncated sequences up to time t, ensuring causality by excluding future information.
• Develop AugmentedOcclusion, which samples baselines from a bootstrapped distribution near the input to reduce out-of-distribution perturbations.
• Integrate BayesLime and BayesKernelShap, which use Bayesian linear regression to provide uncertainty estimates (credible intervals) in explanations.
• Support temporal attribution by returning attribution tensors of shape (T, N) for inputs of shape (T, N), enabling per-time-step interpretability.
• Enable model-agnostic explanation via Non-linearities Tunnel, which replaces ReLU with Softplus in model layers to reduce curvature-induced explanation artifacts.

Figure 1: Illustration of the TIG method . Given an input x and a corresponding baseline $\overline{\textbf{x}}$ , we compute the attribution as such. For each time t, we crop future times: $\textbf{x}[:t]$ , and we only compute interpolations on the last data point in time: $\textbf{x}_{\alpha}=(x_

Experimental results

Research questions

• RQ1 How can we improve the fidelity and robustness of feature attributions in time series models by modifying baseline selection and perturbation strategies?
• RQ2 To what extent do temporal attribution methods like TIG and Time Forward Tunnel provide more accurate and causally consistent explanations than standard attribution techniques?
• RQ3 Can uncertainty-aware methods such as BayesLime and BayesKernelShap improve the reliability of explanations in time series by quantifying explanation confidence?
• RQ4 How effective are similarity measures based on Local Outlier Factor (LOF) in reducing sensitivity to outlying perturbations during local explanation?
• RQ5 To what extent can components of the library be generalized beyond time series, such as for explaining transformer-based language models?

Key findings

• Temporal Integrated Gradients (TIG) provides a causally consistent attribution method by interpolating only the current time step while preserving past inputs and excluding future data.
• Time Forward Tunnel enables any Captum attribution method to be applied in a causal, iterative manner, ensuring that future data does not influence attributions at time t.
• AugmentedOcclusion reduces distributional shift by sampling baselines from a bootstrapped distribution close to the input, improving explanation stability.
• BayesLime and BayesKernelShap provide credible intervals for attributions, enhancing interpretability by quantifying uncertainty in feature importance.
• LOF-enhanced LIME and Kernel SHAP reduce sensitivity to outlying perturbations by using LOF scores as similarity weights, improving robustness in sparse or noisy data.
• The library supports both temporal attribution (per-time-step) and global attribution, with methods like DynaMask, ExtremalMask, and Retain enabling dynamic and context-aware explanation.