[Paper Review] Few-Cost Salient Object Detection with Adversarial-Paced Learning

Dingwen Zhang, Haibin Tian+1 more|arXiv (Cornell University)|Apr 5, 2021

Visual Attention and Saliency Detection10 citations

TL;DR

This paper proposes an adversarial-paced learning (APL) framework for few-cost salient object detection, where only a small number of training images require pixel-wise annotation. By replacing heuristic self-paced regularizers with a data-driven, adversarial mechanism to dynamically assign reliability weights to pseudo-labeled pixels, the method achieves performance comparable to fully supervised models using just 1,000 annotated images.

ABSTRACT

Detecting and segmenting salient objects from given image scenes has received great attention in recent years. A fundamental challenge in training the existing deep saliency detection models is the requirement of large amounts of annotated data. While gathering large quantities of training data becomes cheap and easy, annotating the data is an expensive process in terms of time, labor and human expertise. To address this problem, this paper proposes to learn the effective salient object detection model based on the manual annotation on a few training images only, thus dramatically alleviating human labor in training models. To this end, we name this task as the few-cost salient object detection and propose an adversarial-paced learning (APL)-based framework to facilitate the few-cost learning scenario. Essentially, APL is derived from the self-paced learning (SPL) regime but it infers the robust learning pace through the data-driven adversarial learning mechanism rather than the heuristic design of the learning regularizer. Comprehensive experiments on four widely-used benchmark datasets demonstrate that the proposed method can effectively approach to the existing supervised deep salient object detection models with only 1k human-annotated training images. The project page is available at https://github.com/hb-stone/FC-SOD.

Motivation & Objective

• Address the high annotation cost in deep saliency detection by enabling effective model training with only a few manually annotated images.
• Overcome the limitations of heuristic self-paced learning regularizers that rely on human-designed rules and may underfit or misguide learning in low-data regimes.
• Develop a data-driven learning pace mechanism that adaptively identifies reliable pseudo-labels from unlabeled images to improve model generalization.
• Demonstrate that few-cost learning can achieve performance close to fully supervised models, significantly reducing human annotation burden.

Proposed method

• Propose an adversarial-paced learning (APL) framework derived from self-paced learning (SPL), where the learning pace is inferred via adversarial training instead of heuristic regularizers.
• Introduce a global structure-guided pixel weighting (PW) module to compute reliability scores for image regions based on saliency structure and consistency with ground-truth.
• Formulate the learning objective as a min-max game: a generator produces saliency predictions, while a discriminator distinguishes between real (ground-truth) and fake (pseudo-labeled) masks.
• Use the discriminator’s confidence scores as reliability weights to guide the training of the saliency network, prioritizing high-confidence pseudo-labels.
• Implement an alternative optimization strategy similar to SPL, but with data-driven weight assignment through adversarial feedback.
• Integrate the PW module into the APL framework to preserve structural integrity of salient objects during pseudo-label refinement.

Experimental results

Research questions

• RQ1 Can a data-driven learning pace mechanism outperform heuristic self-paced regularizers in few-cost salient object detection?
• RQ2 How effective is adversarial learning in identifying reliable pseudo-labels from unlabeled images when only a few annotated images are available?
• RQ3 To what extent can a model trained on only 1,000 annotated images match the performance of fully supervised models?
• RQ4 Does the proposed APL framework maintain structural fidelity of salient objects during pseudo-label generation compared to baseline methods?
• RQ5 How robust is the method across varying ratios of labeled data (e.g., 1%, 5%, 10%)?

Key findings

• The proposed APL method achieves a maxF score of 0.863 on the DUTS-TE dataset when trained on only 1,000 annotated images, outperforming existing weakly supervised methods.
• With just 1% of the training data (1,000 images), the model achieves 95.48% of the performance (in maxF) of a fully supervised model trained on all data.
• The ablation study shows that conventional self-paced learning methods fail to improve performance over the baseline without the $γ$-regularizer, indicating poor generalization in low-data regimes.
• The