[Paper Review] Semi-Supervised Segmentation of Concrete Aggregate Using Consensus Regularisation and Prior Guidance

Max Coenen, Thomas Schack+3 more|arXiv (Cornell University)|Apr 22, 2021

Infrastructure Maintenance and MonitoringEngineering25 references15 citations

TL;DR

This paper proposes a semi-supervised semantic segmentation framework for concrete aggregate particles that improves performance on imbalanced datasets by combining consensus regularization with prior-guided losses. By integrating class distribution priors and auto-encoder reconstruction constraints into a lightweight encoder-decoder architecture, the method achieves significant gains in minority class recall—up to 9.8% higher than standard consistency training—while maintaining high overall accuracy on a newly introduced high-resolution concrete core dataset.

ABSTRACT

In order to leverage and profit from unlabelled data, semi-supervised frameworks for semantic segmentation based on consistency training have been proven to be powerful tools to significantly improve the performance of purely supervised segmentation learning. However, the consensus principle behind consistency training has at least one drawback, which we identify in this paper: imbalanced label distributions within the data. To overcome the limitations of standard consistency training, we propose a novel semi-supervised framework for semantic segmentation, introducing additional losses based on prior knowledge. Specifically, we propose a light-weight architecture consisting of a shared encoder and a main decoder, which is trained in a supervised manner. An auxiliary decoder is added as additional branch in order to make use of unlabelled data based on consensus training, and we add additional constraints derived from prior information on the class distribution and on auto-encoder regularisation. Experiments performed on our "concrete aggregate dataset" presented in this paper demonstrate the effectiveness of the proposed approach, outperforming the segmentation results achieved by purely supervised segmentation and standard consistency training.

Motivation & Objective

• To address the limitation of standard consistency training in semi-supervised semantic segmentation when class distributions are imbalanced.
• To improve segmentation performance for concrete aggregate particles, a minority class in sedimentation images, by incorporating prior knowledge about expected label distributions.
• To develop a lightweight, efficient deep learning architecture that achieves state-of-the-art performance with fewer parameters.
• To create and release a new high-resolution benchmark dataset for concrete aggregate segmentation with pixel-level annotations.

Proposed method

• A shared encoder with a main decoder is trained in a supervised manner on limited labeled data.
• An auxiliary decoder is introduced as a second branch to enable consensus training on unlabelled data using data augmentations.
• A prior-guided loss is applied in label space to regularize predictions of unlabelled data based on expected class frequency distributions.
• An auto-encoder reconstruction loss is added in image space to learn the underlying data distribution and improve feature representation.
• The framework combines these regularizations with consistency training, where predictions for augmented views of the same image are encouraged to agree.
• A lightweight architecture based on residual blocks and depthwise separable convolutions is used to reduce model complexity while preserving performance.

Experimental results

Research questions

• RQ1 How does class imbalance affect the performance of standard consistency training in semi-supervised semantic segmentation?
• RQ2 Can prior knowledge about class distribution improve segmentation performance for minority classes in imbalanced datasets?
• RQ3 To what extent do auto-encoder reconstruction losses enhance feature learning and segmentation accuracy in semi-supervised settings?
• RQ4 How does the proposed framework compare to purely supervised and standard consistency training in terms of F1-score and overall accuracy?
• RQ5 What is the impact of varying amounts of labeled data on the performance of the proposed semi-supervised method?

Key findings

• The proposed method achieved a 9.8% increase in recall for the minority class (aggregate) compared to standard consensus training on the Unet architecture.
• On the R-S-Net architecture, the full framework improved aggregate recall by 3.2% and F1-score by 4.1 percentage points over the consensus-only variant.
• The overall accuracy (OA) of the full framework reached 92.8% on the Unet and 92.4% on the R-S-Net, outperforming both supervised and standard semi-supervised baselines.
• The performance gain from semi-supervised learning was most significant when only 10% of the data was labeled (T1 setup), with OA increasing by 4.9% over the supervised baseline.
• The ablation study showed that the largest performance gains occurred between T1 and T3 (tripling labeled data), with OA increasing by 4.9% for the base variant.
• The integration of prior knowledge and auto-encoder regularization effectively reduced false negatives for aggregate particles, as visually confirmed in qualitative results.