[Paper Review] Super-BPD: Super Boundary-to-Pixel Direction for Fast Image Segmentation

Jianqiang Wan, Yang Liu+3 more|arXiv (Cornell University)|May 30, 2020

Medical Image Segmentation Techniques30 references4 citations

TL;DR

This paper proposes Super-BPD, a novel fast image segmentation method that uses boundary-to-pixel direction (BPD) vectors—unit vectors pointing from each pixel’s nearest boundary to itself—to create robust, directionally consistent superpixels. By leveraging direction similarity across adjacent super-BPDs in a region adjacency graph, the method enables efficient, coarse-to-fine segmentation that achieves near real-time inference (~25 fps) with performance comparable or superior to MCG on BSDS500 and Pascal Context.

ABSTRACT

Image segmentation is a fundamental vision task and a crucial step for many applications. In this paper, we propose a fast image segmentation method based on a novel super boundary-to-pixel direction (super-BPD) and a customized segmentation algorithm with super-BPD. Precisely, we define BPD on each pixel as a two-dimensional unit vector pointing from its nearest boundary to the pixel. In the BPD, nearby pixels from different regions have opposite directions departing from each other, and adjacent pixels in the same region have directions pointing to the other or each other (i.e., around medial points). We make use of such property to partition an image into super-BPDs, which are novel informative superpixels with robust direction similarity for fast grouping into segmentation regions. Extensive experimental results on BSDS500 and Pascal Context demonstrate the accuracy and efficency of the proposed super-BPD in segmenting images. In practice, the proposed super-BPD achieves comparable or superior performance with MCG while running at ~25fps vs. 0.07fps. Super-BPD also exhibits a noteworthy transferability to unseen scenes. The code is publicly available at https://github.com/JianqiangWan/Super-BPD.

Motivation & Objective

• To address the inefficiency and inaccuracy of traditional unsupervised image segmentation methods on natural images with weak boundaries.
• To overcome the high computational cost and error propagation in contour-to-segmentation transformation pipelines common in state-of-the-art methods.
• To develop a superpixel-like structure that encodes directional relationships between pixels and boundaries for robust, efficient segmentation.
• To enable fast, transferable segmentation suitable for downstream tasks like object proposal generation without requiring deep learning retraining.
• To achieve a strong trade-off between segmentation accuracy and inference speed, especially in challenging scenarios with weak boundaries.

Proposed method

• Define a boundary-to-pixel direction (BPD) as a 2D unit vector pointing from the nearest boundary to each pixel, encoding both boundary position and relative region membership.
• Partition the image into super-BPDs by grouping pixels with similar BPDs, forming compact, directionally coherent regions that are robust to weak boundaries.
• Construct a region adjacency graph (RAG) where edges between adjacent super-BPDs are weighted by directional similarity along their shared boundary.
• Apply a coarse-to-fine merging strategy on the RAG, using direction similarity to group super-BPDs into final segmentation regions.
• Use a differentiable network to predict BPD vectors end-to-end, with a loss combining L2 and angular domain losses to improve direction accuracy.
• Optimize the number of steps s in direction similarity computation along boundaries, finding s=3 yields optimal performance.

Experimental results

Research questions

• RQ1 Can a boundary-to-pixel direction representation provide a more robust and efficient alternative to classical superpixels for image segmentation?
• RQ2 Does direction similarity between adjacent super-BPDs enable accurate and efficient segmentation, especially at weak boundaries?
• RQ3 Can the proposed method achieve real-time performance while maintaining or exceeding the accuracy of state-of-the-art methods like MCG?
• RQ4 How transferable is the learned BPD representation to unseen scenes and downstream tasks such as object proposal generation?
• RQ5 What hyperparameters (e.g., number of steps s, loss functions) most significantly affect segmentation performance and why?

Key findings

• Super-BPD achieves a region F-measure of 0.472 on the PASCAL Context dataset, outperforming MCG (0.454) and other baselines.
• The method runs at approximately 25 fps on PASCAL Context, significantly faster than MCG’s 0.07 fps, demonstrating near real-time inference.
• The ASPP module improves performance by 0.018 on PASCAL Context and 0.012 on BSDS500, confirming its value in capturing large-scale region structures.
• Combining L2 and angular domain losses yields the best performance (0.472 F_op on PASCAL Context), outperforming either loss alone.
• The optimal number of steps s=3 in direction similarity computation yields the highest accuracy, as it balances directional consistency and local variation.
• In object proposal generation, Super-BPD improves detection recall by nearly twofold over M-BING with DEL until 500 proposals on PASCAL VOC2007.