[Paper Review] Industrial Anomaly Detection and Localization Using Weakly-Supervised Residual Transformers

Hanxi Li, Jingqi Wu+5 more|arXiv (Cornell University)|Jun 6, 2023

Anomaly Detection Techniques and Applications4 citations

TL;DR

This paper proposes SemiREST, a weakly-supervised anomaly detection and localization framework that uses block-wise classification with position-constrained patch-matching residuals and a sliding vision transformer to achieve state-of-the-art performance on MVTec-AD with significantly reduced annotation cost. It outperforms all prior methods, achieving 81.2% AP in unsupervised and 84.4% AP in supervised settings, while even surpassing fully supervised SOTA with only bounding-box annotations.

ABSTRACT

Recent advancements in industrial anomaly detection (AD) have demonstrated that incorporating a small number of anomalous samples during training can significantly enhance accuracy. However, this improvement often comes at the cost of extensive annotation efforts, which are impractical for many real-world applications. In this paper, we introduce a novel framework, Weak}ly-supervised RESidual Transformer (WeakREST), designed to achieve high anomaly detection accuracy while minimizing the reliance on manual annotations. First, we reformulate the pixel-wise anomaly localization task into a block-wise classification problem. Second, we introduce a residual-based feature representation called Positional Fast Anomaly Residuals (PosFAR) which captures anomalous patterns more effectively. To leverage this feature, we adapt the Swin Transformer for enhanced anomaly detection and localization. Additionally, we propose a weak annotation approach, utilizing bounding boxes and image tags to define anomalous regions. This approach establishes a semi-supervised learning context that reduces the dependency on precise pixel-level labels. To further improve the learning process, we develop a novel ResMixMatch algorithm, capable of handling the interplay between weak labels and residual-based representations. On the benchmark dataset MVTec-AD, our method achieves an Average Precision (AP) of $83.0\%$, surpassing the previous best result of $82.7\%$ in the unsupervised setting. In the supervised AD setting, WeakREST attains an AP of $87.6\%$, outperforming the previous best of $86.0\%$. Notably, even when using weaker annotations such as bounding boxes, WeakREST exceeds the performance of leading methods relying on pixel-wise supervision, achieving an AP of $87.1\%$ compared to the prior best of $86.0\%$ on MVTec-AD.

Motivation & Objective

• To address the high annotation cost in industrial anomaly detection while maintaining or improving detection accuracy.
• To reduce reliance on pixel-wise segmentation by reformulating anomaly localization as a block-wise classification problem.
• To enable effective semi-supervised learning using only bounding-box annotations for anomalous regions.
• To develop a novel semi-supervised learning scheme that effectively exploits unlabeled regions from weak labels.
• To achieve state-of-the-art performance on MVTec-AD, BTAD, and KSDD2 with minimal supervision.

Proposed method

• A position-constrained patch-matching (PCF) mechanism generates residuals between test and reference patches, preserving spatial alignment for anomaly sensitivity.
• The anomaly detection task is redefined as block-wise binary classification, where each block is labeled as normal, positive (anomalous), or ignored (unlabeled), reducing annotation burden.
• A sliding vision transformer (Swin Transformer) processes these block-level features to predict anomaly scores with high spatial precision.
• A bagging strategy across multiple Swin Transformer heads improves robustness and generalization in the prediction pipeline.
• A customized MixMatch-based semi-supervised learning scheme leverages unlabeled regions from bounding-box annotations, using two novel data augmentation techniques: K-NN-based augmentation and random dropout.
• The method uses a square distance function on residuals in unsupervised and semi-supervised regimes for better feature selection, while absolute difference is used when real anomalies are available.

Experimental results

Research questions

• RQ1 Can block-wise classification reduce annotation cost in anomaly detection without sacrificing accuracy?
• RQ2 Can a sliding vision transformer effectively localize anomalies using only residuals from position-constrained patch-matching?
• RQ3 Can semi-supervised learning with bounding-box annotations achieve performance comparable to full-supervision?
• RQ4 Does a customized MixMatch scheme effectively exploit unlabeled regions in weakly-supervised anomaly detection?
• RQ5 Can the proposed method outperform existing SOTA approaches in both unsupervised and semi-supervised settings?

Key findings

• SemiREST achieves 81.2% Average Precision (AP) on MVTec-AD in the unsupervised setting, surpassing the previous SOTA of 75.8% AP.
• In the fully supervised setting, SemiREST achieves 84.4% AP, exceeding the prior SOTA of 78.6% AP.
• With only bounding-box annotations, SemiREST attains 83.8% AP, outperforming the previous SOTA with full pixel-wise supervision (78.6% AP).
• The method maintains strong performance across three benchmark datasets: MVTec-AD, BTAD, and KSDD2, demonstrating generalization.
• Ablation studies confirm that each component—PCF, bagging, K-NN augmentation, MixMatch, and random dropout—contributes to performance gains.
• The square distance function outperforms absolute difference in unsupervised and semi-supervised settings due to its feature-selection property.