[Paper Review] SemiContour: A Semi-supervised Learning Approach for Contour Detection

Zizhao Zhang, Fuyong Xing+2 more|arXiv (Cornell University)|May 17, 2016

Industrial Vision Systems and Defect Detection30 references13 citations

TL;DR

This paper proposes SemiContour, a semi-supervised structured ensemble learning method for contour detection that leverages sparse representation to embed unlabeled image patches into a discriminative low-dimensional subspace, enabling effective node splitting in structured random forests (SRF) with only three labeled images. The method achieves state-of-the-art performance on BSDS500 and NYU Depth datasets, outperforming fully supervised methods trained on hundreds of images.

ABSTRACT

Supervised contour detection methods usually require many labeled training images to obtain satisfactory performance. However, a large set of annotated data might be unavailable or extremely labor intensive. In this paper, we investigate the usage of semi-supervised learning (SSL) to obtain competitive detection accuracy with very limited training data (three labeled images). Specifically, we propose a semi-supervised structured ensemble learning approach for contour detection built on structured random forests (SRF). To allow SRF to be applicable to unlabeled data, we present an effective sparse representation approach to capture inherent structure in image patches by finding a compact and discriminative low-dimensional subspace representation in an unsupervised manner, enabling the incorporation of abundant unlabeled patches with their estimated structured labels to help SRF perform better node splitting. We re-examine the role of sparsity and propose a novel and fast sparse coding algorithm to boost the overall learning efficiency. To the best of our knowledge, this is the first attempt to apply SSL for contour detection. Extensive experiments on the BSDS500 segmentation dataset and the NYU Depth dataset demonstrate the superiority of the proposed method.

Motivation & Objective

• Address the challenge of limited labeled data in supervised contour detection, where acquiring large annotated datasets is costly and time-consuming.
• Explore the feasibility and effectiveness of semi-supervised learning (SSL) for contour detection, a domain previously unexplored in this context.
• Improve node splitting in structured random forests (SRF) by incorporating unlabeled image patches through unsupervised sparse representation.
• Develop a fast and efficient sparse coding algorithm to scale the method to large-scale datasets while maintaining high accuracy.
• Demonstrate strong generalization ability by leveraging unlabeled data from different domains, such as combining NYU Depth and BSDS500 data.

Proposed method

• Propose a semi-supervised structured ensemble learning framework built on structured random forests (SRF), where unlabeled patches (u-tokens) are used to improve node splitting in the presence of only a few labeled patches (l-tokens).
• Introduce an unsupervised sparse representation technique to learn a compact, discriminative low-dimensional subspace embedding of image patches, enabling the use of unlabeled data in SRF training.
• Apply a novel fast sparse coding algorithm to efficiently compute sparse codes, with time complexity dominated by O(d·V), where d is feature dimension and V is dictionary size.
• Leverage the max-margin properties of SRF to dynamically estimate structured labels for u-tokens within tree nodes, enhancing decision boundary learning.
• Use the estimated structured labels of u-tokens to guide the splitting of internal nodes in SRF, improving generalization and reducing overfitting on limited labeled data.
• Integrate the sparse representation and label estimation process directly into the SRF training pipeline, allowing end-to-end learning with minimal supervision.

Experimental results

Research questions

• RQ1 Can semi-supervised learning be effectively applied to contour detection, where labeled data is scarce?
• RQ2 Can unlabeled image patches be meaningfully incorporated into structured random forests to improve node splitting and detection accuracy?
• RQ3 Does learning a discriminative low-dimensional subspace via unsupervised sparse representation enhance contour detection performance with minimal labeled data?
• RQ4 Can a fast sparse coding algorithm be designed to scale the method to large datasets without sacrificing accuracy?
• RQ5 Does the method generalize well across domains when unlabeled data from a different dataset (e.g., NYU Depth) is used during training?

Key findings

• With only three labeled images, SemiContour achieves detection performance comparable to or better than fully supervised methods trained on hundreds of images.
• On the BSDS500 dataset, SemiContour with 10 labeled images outperforms supervised SRF (SE-Var) trained on 10 images and matches performance of fully supervised models trained on all 381 images.
• On the NYU Depth dataset, SemiContour achieves ODS = 0.68, OIS = 0.70, and AP = 0.69 with only 10 labeled images, surpassing supervised baselines trained on the same data.
• In cross-dataset generalization, SemiContour achieves ODS = 0.73 and OIS = 0.75 on NYUD using BSDS500 as unlabeled data, outperforming SE-Var in both settings.
• The proposed fast sparse coding algorithm is significantly faster than OMP, Batch-OMP, and Batch-OMP-fast, especially as the number of test samples increases, while maintaining equivalent detection accuracy.
• The method demonstrates strong robustness and noise resistance, even when incorporating unlabeled patches from a different domain (e.g., indoor NYU scenes into outdoor BSDS500 training).