[Paper Review] Visual Anomaly Detection Via Partition Memory Bank Module and Error Estimation

Peng Xing, Zechao Li|arXiv (Cornell University)|Sep 26, 2022

Anomaly Detection Techniques and Applications4 citations

TL;DR

This paper proposes a novel unsupervised visual anomaly detection method using a Partition Memory Bank (PMB) module and a Histogram Error Estimation module. The PMB learns and stores semantically intact normal features via a unique partition-based query generation method, while the error estimation module adaptively eliminates cumulative reconstruction errors, significantly improving both anomaly detection and localization performance on standard benchmarks.

ABSTRACT

Reconstruction method based on the memory module for visual anomaly detection attempts to narrow the reconstruction error for normal samples while enlarging it for anomalous samples. Unfortunately, the existing memory module is not fully applicable to the anomaly detection task, and the reconstruction error of the anomaly samples remains small. Towards this end, this work proposes a new unsupervised visual anomaly detection method to jointly learn effective normal features and eliminate unfavorable reconstruction errors. Specifically, a novel Partition Memory Bank (PMB) module is proposed to effectively learn and store detailed features with semantic integrity of normal samples. It develops a new partition mechanism and a unique query generation method to preserve the context information and then improves the learning ability of the memory module. The proposed PMB and the skip connection are alternatively explored to make the reconstruction of abnormal samples worse. To obtain more precise anomaly localization results and solve the problem of cumulative reconstruction error, a novel Histogram Error Estimation module is proposed to adaptively eliminate the unfavorable errors by the histogram of the difference image. It improves the anomaly localization performance without increasing the cost. To evaluate the effectiveness of the proposed method for anomaly detection and localization, extensive experiments are conducted on three widely-used anomaly detection datasets. The encouraging performance of the proposed method compared to the recent approaches based on the memory module demonstrates its superiority.

Motivation & Objective

• Address the limitation of existing memory modules in visual anomaly detection, where anomalous samples still achieve low reconstruction error due to logical query reorganization that distorts semantic integrity.
• Improve the learning of normal features by preserving spatial and channel-wise context through a novel partition mechanism and query generation strategy in the PMB module.
• Reduce cumulative reconstruction error in autoencoder-based models that hinders accurate anomaly localization, especially for subtle or small-region anomalies.
• Enhance both anomaly detection and localization performance without increasing inference cost by introducing a histogram-based error estimation mechanism.
• Optimize the integration of the PMB module with skip connections to balance high-level semantic consistency and detailed feature reconstruction.

Proposed method

• Introduce the Partition Memory Bank (PMB) module, which divides feature maps into spatial partitions and generates queries that preserve semantic and contextual information from the original feature map.
• Design a unique query generation method that avoids logical reorganization across channels, ensuring the memory module learns actual normal features rather than abstracted logical patterns.
• Alternate the use of the PMB module and skip connections during the reconstruction process to force abnormal features to reconstruct poorly by relying on stored normal features.
• Propose a Histogram Error Estimation module that analyzes the distribution of reconstruction error differences in normal regions, enabling adaptive elimination of unfavorable errors.
• Use the histogram of the difference image between input and reconstructed features to dynamically adjust error weights, improving localization precision without additional parameters.
• Optimize hyperparameters such as the number of partitions (k), the number of PMB layers (L), and the number of iterations (p) to balance detection and localization performance.

Figure 1: Illustration of the query generation methods for different memory modules. Figures (1) [ 10 ] and (2) [ 11 ] represent the logical query generated from the same position in the feature map, with different channels. Figure (3) represents the proposed query. It preserves the critical feature

Experimental results

Research questions

• RQ1 Can a memory module that preserves semantic integrity of normal features through spatial partitioning improve reconstruction fidelity for normal samples and increase error for anomalous samples?
• RQ2 How does the proposed PMB module compare to traditional memory modules in terms of feature representation quality and anomaly detection capability?
• RQ3 To what extent can the Histogram Error Estimation module reduce cumulative reconstruction error and improve localization accuracy in autoencoder-based anomaly detection?
• RQ4 What is the optimal configuration of the PMB module (e.g., number of partitions k, number of layers L, iteration count p) for balancing detection and localization performance?
• RQ5 Does the joint use of PMB and skip connections enhance both reconstruction quality and anomaly discrimination in a way that is robust across diverse datasets?

Key findings

• The proposed method achieves state-of-the-art performance on three benchmark datasets: MVTec AD, MNIST, and Retinal-OCT, with 98.1% AUC on MNIST and 98.0% on Retinal-OCT.
• On the MVTec AD dataset, the method achieves 97.6% AUC with k=(4,4,4), demonstrating superior detection performance, especially for texture and fine-grained anomalies.
• The use of skip connections in combination with PMB improves detection performance by 6% compared to a baseline without skip connections, particularly benefiting texture-rich images.
• Smaller values of k (e.g., k=(2,2,2)) yield better localization results due to stronger error amplification, while larger k improves detection by enabling more detailed feature learning.
• The Histogram Error Estimation module effectively reduces cumulative reconstruction errors, enabling high-precision localization even in normal regions with subtle anomalies.
• The method outperforms recent memory-based approaches by jointly optimizing feature learning and error correction, proving effective across diverse data distributions and anomaly types.

Figure 2: The differences in query generation method and memory item between the proposed method and the traditional method. ”ERROR” represents reconstruction error. The memory modules are both learned from the reconstruction task of category A to category A using reconstruction loss update. However