[Paper Review] Explaining Anomalies Detected by Autoencoders Using SHAP

Liat Antwarg, Miller, Ronnie Mindlin+2 more|arXiv (Cornell University)|Mar 6, 2019

Anomaly Detection Techniques and ApplicationsComputer Science36 references83 citations

TL;DR

The paper proposes a model-agnostic Kernel SHAP method to explain anomalies detected by autoencoders, linking high reconstruction errors to the most influential features, and validates with real-world user studies and synthetic data.

ABSTRACT

Anomaly detection algorithms are often thought to be limited because they don't facilitate the process of validating results performed by domain experts. In Contrast, deep learning algorithms for anomaly detection, such as autoencoders, point out the outliers, saving experts the time-consuming task of examining normal cases in order to find anomalies. Most outlier detection algorithms output a score for each instance in the database. The top-k most intense outliers are returned to the user for further inspection; however the manual validation of results becomes challenging without additional clues. An explanation of why an instance is anomalous enables the experts to focus their investigation on most important anomalies and may increase their trust in the algorithm. Recently, a game theory-based framework known as SHapley Additive exPlanations (SHAP) has been shown to be effective in explaining various supervised learning models. In this research, we extend SHAP to explain anomalies detected by an autoencoder, an unsupervised model. The proposed method extracts and visually depicts both the features that most contributed to the anomaly and those that offset it. A preliminary experimental study using real world data demonstrates the usefulness of the proposed method in assisting the domain experts to understand the anomaly and filtering out the uninteresting anomalies, aiming at minimizing the false positive rate of detected anomalies.

Motivation & Objective

• Motivate the need for per-instance explanations to increase expert trust in autoencoder-based anomaly detection.
• Develop a black-box explanation method that works without knowing the internal autoencoder architecture.
• Link high reconstruction errors to the features most responsible for anomaly scores.
• Provide visual and tabular explanations that separate contributing and offsetting features.
• Evaluate the explanations through user studies, synthetic ground-truth experiments, robustness tests, and anomaly score manipulation.

Proposed method

• Compute reconstruction error L(X,X') as the sum of squared feature-wise errors.
• Identify topMfeatures with highest per-feature reconstruction error to focus explanations.
• Use Kernel SHAP to compute SHAP values for each top feature with respect to predicting its reconstructed value X' i.
• Split SHAP values into contributing (pushing prediction away from true value) and offsetting (pushing toward the true value) features using the polarity of SHAP values and the comparison between X and X'.
• Present explanations as a colored table showing contributing (red) and offsetting (blue) features for each top feature, with SHAP value magnitudes indicating importance.
• Optionally compare against an alternative approach where SHAP explains total reconstruction error via an extra layer, and confirm consistency of top features.

Experimental results

Research questions

• RQ1 Can Kernel SHAP provide reliable, model-agnostic explanations for anomalies detected by an autoencoder?
• RQ2 Which features and their interactions best explain high reconstruction errors in autoencoder outputs?
• RQ3 Do SHAP-based explanations more accurately reflect true contributing factors than other methods like LIME in this context?
• RQ4 Do explanations improve domain experts’ understanding and inspection efficiency of anomalies in real-world data?

Key findings

• The proposed SHAP-based explanations reveal both contributing and offsetting features for anomalies detected by autoencoders.
• Domain experts reported that visual explanations helped focus on the most important explanatory features for inspection.
• In synthetic ground-truth tests, explanations used SHAP correctly identified the exact features responsible for anomalies.
• The SHAP-based explanations were more robust than LIME in the evaluated settings.
• The explanations were effective at reducing the anomaly score when used to manipulate explanatory features in the experiments.
• Across real-world datasets, the method supported better interpretability without requiring knowledge of the autoencoder internals.