[Paper Review] DeSTSeg: Segmentation Guided Denoising Student-Teacher for Anomaly Detection

Xuan Zhang, Shiyu Li+4 more|arXiv (Cornell University)|Nov 21, 2022

Anomaly Detection Techniques and Applications4 citations

TL;DR

DeSTSeg proposes a denoising student-teacher framework for anomaly detection that enhances feature discrimination by training a student network to reconstruct clean features from synthetically corrupted inputs, while using a trainable segmentation network to adaptively fuse multi-level feature discrepancies. The method achieves state-of-the-art performance with 98.6% image-level AUC, 75.8% pixel-level AP, and 76.4% instance-level AP on the MVTec AD benchmark.

ABSTRACT

Visual anomaly detection, an important problem in computer vision, is usually formulated as a one-class classification and segmentation task. The student-teacher (S-T) framework has proved to be effective in solving this challenge. However, previous works based on S-T only empirically applied constraints on normal data and fused multi-level information. In this study, we propose an improved model called DeSTSeg, which integrates a pre-trained teacher network, a denoising student encoder-decoder, and a segmentation network into one framework. First, to strengthen the constraints on anomalous data, we introduce a denoising procedure that allows the student network to learn more robust representations. From synthetically corrupted normal images, we train the student network to match the teacher network feature of the same images without corruption. Second, to fuse the multi-level S-T features adaptively, we train a segmentation network with rich supervision from synthetic anomaly masks, achieving a substantial performance improvement. Experiments on the industrial inspection benchmark dataset demonstrate that our method achieves state-of-the-art performance, 98.6% on image-level AUC, 75.8% on pixel-level average precision, and 76.4% on instance-level average precision.

Motivation & Objective

• To address the limitation of existing student-teacher frameworks in generating discriminative features for anomalous samples due to lack of explicit constraints on anomalies.
• To improve multi-level feature fusion in anomaly detection by replacing empirical aggregation with a trainable segmentation network.
• To enhance robustness and representation learning by introducing a denoising training objective using synthetic anomalies.
• To achieve state-of-the-art performance across image-level, pixel-level, and instance-level anomaly detection benchmarks.

Proposed method

• A denoising student encoder-decoder network is trained to reconstruct clean features from the teacher network using synthetically corrupted normal images as input.
• The student network is trained to minimize the feature discrepancy between its output and the teacher's output on the same image, effectively learning to denoise features.
• A segmentation network is trained with rich supervision from synthetic anomaly masks to adaptively fuse multi-level feature discrepancies from the student-teacher networks.
• The input to the segmentation network is the element-wise product of normalized feature maps from the student and teacher networks, balancing feature information and prior knowledge.
• The segmentation network uses L1 loss to supervise the prediction of anomaly score maps, improving localization accuracy.
• The entire framework integrates a pre-trained teacher, a denoising student, and a trainable segmentation head into an end-to-end training pipeline.

Experimental results

Research questions

• RQ1 Can a denoising training objective improve feature discrimination between normal and anomalous samples in a student-teacher framework?
• RQ2 Does replacing empirical multi-level feature fusion with a trainable segmentation network lead to better anomaly localization?
• RQ3 How effective is the use of synthetic anomaly masks in supervising the segmentation network for improved performance?
• RQ4 What is the contribution of each component—denosing, encoder-decoder architecture, and segmentation head—to overall performance?

Key findings

• DeSTSeg achieves 98.6% image-level AUC on the MVTec AD benchmark, outperforming prior state-of-the-art methods by 0.1%.
• The method achieves 75.8% pixel-level average precision, representing a 5.6% improvement over previous SOTA.
• Instance-level average precision reaches 76.4%, a 4.9% improvement over the previous best method.
• Abllation studies confirm that combining denoising training, encoder-decoder architecture, and the segmentation head yields the best performance.
• The L1 loss in the segmentation head improves performance, with 98.6% AUC and 75.8% AP when used.
• The element-wise product of normalized S-T features proves more effective than concatenation or cosine distance as input to the segmentation network.