[Paper Review] Boundary-Aware Segmentation Network for Mobile and Web Applications

Xuebin Qin, Deng-Ping Fan+7 more|arXiv (Cornell University)|Jan 12, 2021

Visual Attention and Saliency Detection127 references69 citations

TL;DR

BASNet introduces a boundary-aware image segmentation network with a predict-refine architecture and a hybrid loss (BCE+SSIM+IoU) to achieve sharp boundaries for salient and camouflaged objects, running at over 70 fps and enabling AR Copy & Paste and Object Cut apps.

ABSTRACT

Although deep models have greatly improved the accuracy and robustness of image segmentation, obtaining segmentation results with highly accurate boundaries and fine structures is still a challenging problem. In this paper, we propose a simple yet powerful Boundary-Aware Segmentation Network (BASNet), which comprises a predict-refine architecture and a hybrid loss, for highly accurate image segmentation. The predict-refine architecture consists of a densely supervised encoder-decoder network and a residual refinement module, which are respectively used to predict and refine a segmentation probability map. The hybrid loss is a combination of the binary cross entropy, structural similarity and intersection-over-union losses, which guide the network to learn three-level (ie, pixel-, patch- and map- level) hierarchy representations. We evaluate our BASNet on two reverse tasks including salient object segmentation, camouflaged object segmentation, showing that it achieves very competitive performance with sharp segmentation boundaries. Importantly, BASNet runs at over 70 fps on a single GPU which benefits many potential real applications. Based on BASNet, we further developed two (close to) commercial applications: AR COPY & PASTE, in which BASNet is integrated with augmented reality for "COPYING" and "PASTING" real-world objects, and OBJECT CUT, which is a web-based tool for automatic object background removal. Both applications have already drawn huge amount of attention and have important real-world impacts. The code and two applications will be publicly available at: https://github.com/NathanUA/BASNet.

Motivation & Objective

• Improve the spatial accuracy of image segmentation boundaries and fine structures.
• Propose a simple yet effective predict-refine architecture combining a deep encoder-decoder with a residual refinement module.
• Design a hybrid loss that fuses BCE, SSIM, and IoU to supervise pixel-, patch-, and map-level representations.
• Demonstrate strong performance on salient object segmentation and camouflaged object segmentation while enabling real-world applications.

Proposed method

• Propose BASNet with a densely supervised encoder-decoder prediction module and a residual refinement module that refines the coarse map.
• Use a hybrid loss combining BCE, SSIM, and IoU to supervise eight outputs across pixel-, patch-, and map-level representations.
• Implement a deep supervision scheme where each decoder stage and an additional refinement module produce segmentation maps.
• Balance boundary fidelity and global region accuracy via the three-level loss.
• Provide two near-commercial applications (AR Copy & Paste and Object Cut) built on BASNet to demonstrate practicality.

Experimental results

Research questions

• RQ1 Can BASNet achieve sharp, accurate boundaries for both salient and camouflaged objects?
• RQ2 How does the proposed predict-refine architecture with a hybrid loss compare to standard encoder-decoder and single-loss approaches on boundary and regional metrics?
• RQ3 What is the impact of the residual refinement module on final segmentation quality and boundary precision?
• RQ4 Do BASNet-based applications (AR Copy & Paste, Object Cut) offer practical, real-time performance for real-world use cases?

Key findings

• BASNet achieves competitive performance on six salient object segmentation datasets and on CAM (COD) datasets, with superior boundary evaluation metrics.
• The model runs at over 70 frames per second on a single GPU, enabling real-time or near-real-time applications.
• The hybrid loss (BCE+SSIM+IoU) over pixel-, patch-, and map-level targets effectively emphasizes precise boundaries and fine structures.
• A densely supervised encoder-decoder prediction module paired with a residual refinement module substantially improves segmentation quality over baselines.
• Two BASNet-based applications, AR Copy & Paste and Object Cut, illustrate practical deployment potential and user impact.