[Paper Review] Analysis on DeepLabV3+ Performance for Automatic Steel Defects Detection

Zheng Nie, Jiachen Xu+1 more|arXiv (Cornell University)|Apr 9, 2020

Industrial Vision Systems and Defect Detection12 references6 citations

TL;DR

This paper evaluates DeepLabV3+ with ResNet101, EfficientNet-B1, and DenseNet201 backbones for automatic steel defect segmentation on the Severstal dataset. Using random weighted data augmentation to address class imbalance, the authors achieve a mean IoU of 0.57 with ResNet101 and EfficientNet-B1, outperforming DenseNet201 (0.325), while ResNet101 also shows the fastest training time.

ABSTRACT

Our works experimented DeepLabV3+ with different backbones on a large volume of steel images aiming to automatically detect different types of steel defects. Our methods applied random weighted augmentation to balance different defects types in the training set. And then applied DeeplabV3+ model three different backbones, ResNet, DenseNet and EfficientNet, on segmenting defection regions on the steel images. Based on experiments, we found that applying ResNet101 or EfficientNet as backbones could reach the best IoU scores on the test set, which is around 0.57, comparing with 0.325 for using DenseNet. Also, DeepLabV3+ model with ResNet101 as backbone has the fewest training time.

Motivation & Objective

• To investigate the performance of DeepLabV3+ with different backbones (ResNet, DenseNet, EfficientNet) for automatic steel defect detection in industrial settings.
• To address the severe class imbalance in steel defect datasets through a custom random weighted data augmentation strategy.
• To evaluate model performance in terms of mean Intersection over Union (mIoU) and training efficiency on real-world steel surface images.
• To analyze model limitations, particularly regarding small or fragmented defects that are hard to detect or label manually.

Proposed method

• Applied DeepLabV3+ with three pre-trained backbones—ResNet101, DenseNet201, and EfficientNet-B1—on a large-scale steel defect dataset from the Severstal Kaggle competition.
• Used random weighted data augmentation to balance defect class frequencies, where augmentation probability was inversely proportional to class frequency.
• Applied three augmentation types—random crop, vertical flip, and random rotation—with class-specific weights to preserve spatial and semantic consistency in augmented images and masks.
• Performed image resizing from 256×1700 to 256×256 for training, while inference was also tested on original resolution (256×1700) for comparison.
• Evaluated models using mean Intersection over Union (mIoU), computed as the average of IoU scores across all test samples.
• Conducted ablation studies comparing a bare-bones DeepLabV3+ baseline with full encoder-decoder models to assess the impact of the encoder module.

Experimental results

Research questions

• RQ1 How do different backbone architectures (ResNet101, DenseNet201, EfficientNet-B1) affect the mIoU performance of DeepLabV3+ in steel defect segmentation?
• RQ2 To what extent does random weighted data augmentation improve model generalization on imbalanced steel defect datasets?
• RQ3 Which backbone provides the best trade-off between segmentation accuracy (mIoU) and training efficiency?
• RQ4 Why do some models achieve low IoU scores despite high mIoU on average, and what role does data imbalance or annotation quality play?
• RQ5 Can the model detect defects smaller than human-annotated thresholds, and how does this affect mIoU evaluation?

Key findings

• ResNet101 and EfficientNet-B1 backbones achieved the highest mIoU scores of approximately 0.57, significantly outperforming the baseline (0.29) and DenseNet201 (0.325).
• ResNet101 demonstrated the fastest training time among all backbones, making it the most efficient choice for real-time industrial deployment.
• DenseNet201 performed poorly, achieving only 0.325 mIoU, likely due to its architecture's sensitivity to small and fragmented defects common in class 1 and 2.
• Low IoU scores were frequently linked to small or highly fragmented defects (especially class 1), which are underrepresented in both count and area.
• The model detected some defects too small for human annotation, which were not labeled in the ground truth but were correctly predicted, suggesting potential for improved labeling in future.
• Visual analysis revealed that the encoder-decoder structure significantly improved edge localization and segmentation precision compared to the baseline model.