[Paper Review] PASCAL Boundaries: A Class-Agnostic Semantic Boundary Dataset

Vittal Premachandran, Boyán Bonev+1 more|arXiv (Cornell University)|Nov 25, 2015

Advanced Neural Network Applications29 references3 citations

TL;DR

This paper introduces the PASCAL Boundaries dataset, a large-scale, class-agnostic semantic boundary dataset with ~10,000 images and 459 semantic classes, addressing ambiguity in traditional edge detection. It proposes a multi-scale deep neural network, M-DSBD, which achieves an F-measure of 0.652 on the new dataset, outperforming baselines and demonstrating strong transferability to BSDS500.

ABSTRACT

In this paper, we address the boundary detection task motivated by the ambiguities in current definition of edge detection. To this end, we generate a large database consisting of more than 10k images (which is 20x bigger than existing edge detection databases) along with ground truth boundaries between 459 semantic classes including both foreground objects and different types of background, and call it the PASCAL Boundaries dataset, which will be released to the community. In addition, we propose a novel deep network-based multi-scale semantic boundary detector and name it Multi-scale Deep Semantic Boundary Detector (M-DSBD). We provide baselines using models that were trained on edge detection and show that they transfer reasonably to the task of boundary detection. Finally, we point to various important research problems that this dataset can be used for.

Motivation & Objective

• Address the ambiguity in traditional edge detection by defining a precise, class-agnostic boundary detection task focused on semantic instance-level boundaries.
• Construct a large-scale, high-quality dataset with 10,000 images and 459 semantic classes to support boundary detection research.
• Develop a novel deep learning architecture, M-DSBD, for multi-scale semantic boundary detection to improve performance over existing edge detection models.
• Provide strong baselines using models pre-trained on BSDS500 to enable fair comparison and demonstrate transferability to the new dataset.
• Enable new research directions in joint semantic segmentation and boundary detection, object proposal generation, and occlusion reasoning.

Proposed method

• Annotate boundaries between 459 semantic classes (foreground objects and background types) in 10,000 images from PASCAL VOC2010, focusing only on object outlines and not internal or texture edges.
• Propose a multi-scale deep semantic boundary detector (M-DSBD) that processes feature maps at multiple scales (1, 0.8, 0.5) and fuses predictions using learnable weights.
• Train the model using a boundary detection loss function (Eq. 7) that backpropagates gradients from the original image size, improving feature learning.
• Use side-output supervision from multiple network layers, but treat them as feature extractors rather than independent detectors, improving fusion performance.
• Train the M-DSBD model end-to-end with multi-scale inputs, updating only scale-specific side-output and fusion weights while freezing the base network during final training.
• Compare performance using standard metrics: ODS (optimal threshold F-measure), OIS (optimal intersection over union F-measure), and AP (average precision).

Experimental results

Research questions

• RQ1 Can a large-scale, class-agnostic semantic boundary dataset reduce ambiguity in edge detection compared to traditional edge detection datasets?
• RQ2 How does multi-scale feature fusion improve boundary detection performance compared to single-scale models?
• RQ3 To what extent can models trained on edge detection datasets (e.g., BSDS500) generalize to a new boundary detection task with different annotation semantics?
• RQ4 How does the proposed M-DSBD model compare to state-of-the-art edge detectors (e.g., HED, SE) on a new, larger, and more consistent boundary detection benchmark?
• RQ5 What are the implications of using semantic boundaries for downstream tasks such as object proposal generation and occlusion reasoning?

Key findings

• The PASCAL Boundaries dataset contains 10,000 images with 459 semantic classes, making it 20× larger than BSDS500 and free from the ambiguity of multi-granularity edges.
• The proposed M-DSBD model achieves an F-measure of 0.652 on the PASCAL Boundaries dataset, outperforming the single-scale DSBD (0.643) and all baseline edge detectors.
• The M-DSBD model improves performance by 1% over the single-scale version through multi-scale feature fusion, confirming the benefit of scale-aware boundary detection.
• The M-DSBD model generalizes well to BSDS500, achieving an ODS F-measure of 0.751, outperforming SE (0.746) and HED (0.782) on that dataset when trained on PASCAL Boundaries.
• Qualitative results show that M-DSBD detects only semantic instance boundaries with high confidence and suppresses internal edges, unlike traditional edge detectors.
• The model demonstrates strong transferability, indicating that learning from semantic boundaries improves generalization to edge detection tasks, even when the source and target datasets differ in annotation style.