[Paper Review] Unsupervised Pixel-level Road Defect Detection via Adversarial Image-to-Frequency Transform

Jongmin Yu, Du Yong Kim+2 more|arXiv (Cornell University)|Jan 30, 2020

Infrastructure Maintenance and Monitoring32 references4 citations

TL;DR

This paper proposes Adversarial Image-to-Frequency Transform (AIFT), an unsupervised method for pixel-level road defect detection that leverages adversarial learning to map images to frequency domains without requiring defect annotations. By training on normal road images alone, AIFT achieves state-of-the-art performance on multiple benchmark datasets, outperforming both supervised and unsupervised methods in most cases, with a 3% improvement over prior SOTA on GAPs384 and 8.8% on Cracktree200.

ABSTRACT

In the past few years, the performance of road defect detection has been remarkably improved thanks to advancements on various studies on computer vision and deep learning. Although a large-scale and well-annotated datasets enhance the performance of detecting road pavement defects to some extent, it is still challengeable to derive a model which can perform reliably for various road conditions in practice, because it is intractable to construct a dataset considering diverse road conditions and defect patterns. To end this, we propose an unsupervised approach to detecting road defects, using Adversarial Image-to-Frequency Transform (AIFT). AIFT adopts the unsupervised manner and adversarial learning in deriving the defect detection model, so AIFT does not need annotations for road pavement defects. We evaluate the efficiency of AIFT using GAPs384 dataset, Cracktree200 dataset, CRACK500 dataset, and CFD dataset. The experimental results demonstrate that the proposed approach detects various road detects, and it outperforms existing state-of-the-art approaches.

Motivation & Objective

• Address the challenge of limited real-world diversity in road defect datasets, which hinders the generalization of supervised models.
• Overcome the limitations of reconstruction-based unsupervised methods that may fail to detect defects due to overfitting to unseen patterns.
• Develop a robust, annotation-free approach for detecting diverse road defects under varying conditions using adversarial learning.
• Improve detection reliability by leveraging domain transformation between image and frequency spaces, enhancing model generalization without ground-truth labels.

Proposed method

• Train a generative model to map normal road images to the frequency domain using adversarial learning, with a generator G and two discriminators (DI and DF).
• Implement a cycle-consistency loss to ensure that frequency-to-image and image-to-frequency reconstructions preserve structural integrity.
• Use a total loss combining adversarial, cycle-consistency, and perceptual losses to optimize the generator and discriminators jointly.
• Detect defects by measuring reconstruction inconsistency: high error in the image-to-frequency or frequency-to-image cycle indicates potential defects.
• Apply a threshold-based anomaly detection mechanism on the reconstruction error maps to localize defects without any pixel-level annotations.
• Leverage frequency-domain representation to enhance feature discrimination, making the model robust to lighting and weather variations.

Figure 1 : Architectural detail of the adversarial image-to-frequency transform. The blue objects denote the operation units including the generator $G$ and the discriminators $\mathcal{D}^{I}$ and $\mathcal{D}^{F}$ . The red circles indicate the loss functions corresponded to the each operation uni

Experimental results

Research questions

• RQ1 Can adversarial learning in the image-to-frequency domain improve unsupervised road defect detection beyond reconstruction-based methods?
• RQ2 Does domain transformation between image and frequency spaces enhance robustness to real-world variations like lighting and weather?
• RQ3 Can a model trained only on normal road images detect diverse defect patterns without any annotated defect data?
• RQ4 How does AIFT compare to state-of-the-art supervised and unsupervised methods in terms of anomaly detection performance across multiple benchmarks?
• RQ5 Does the use of adversarial loss improve detection generalization compared to purely reconstruction-based training?

Key findings

• AIFT trained with the total loss (AIFT_total) achieved an AIU of 0.083, ODS of 0.249, and OIS of 0.247 on the GAPs384 dataset, outperforming FPHBN (SOTA at the time) by 3% in ODS and OIS.
• On the Cracktree200 dataset, AIFT_total achieved an AIU of 0.045, ODS of 0.607, and OIS of 0.642, showing an 8.8% improvement over prior unsupervised methods.
• On the CFD dataset, AIFT_total achieved an AIU of 0.203, ODS of 0.701, and OIS of 0.732, demonstrating superior performance compared to existing unsupervised and supervised baselines.
• AIFT_total outperformed reconstruction-based AIFT (AIFT_re) across all datasets, confirming that adversarial training enhances detection robustness and generalization.
• Despite not achieving the highest score on CRACK500 (where FPHBN led with 0.489 AIU), AIFT_total still achieved 0.478 AIU, showing strong performance without any annotated training data.
• The AIU trends in Figure 4 confirm that adversarially trained AIFT models generalize better than reconstruction-based counterparts, especially in detecting subtle or rare defect patterns.

Figure 2 : Structural details of the network models in the generator $G$ and the discriminators $\mathcal{D}^{I}$ and $\mathcal{D}^{F}$ . (a) and (b) denote the structural details of the generator $G$ and the two discriminators $\mathcal{D}^{I}$ and $\mathcal{D}^{F}$ , respectively. The green, blue,