[论文解读] Boundary-Aware Segmentation Network for Mobile and Web Applications

Xuebin Qin, Deng-Ping Fan+7 more|arXiv (Cornell University)|Jan 12, 2021

Visual Attention and Saliency Detection参考文献 127被引用 69

一句话总结

BASNet 引入一个边界感知的图像分割网络，采用 predict-refine 架构和混合损失（BCE+SSIM+IoU），以获得显著对象和伪装对象的清晰边界，运行速度超过 70 fps，并支持 AR Copy & Paste 与 Object Cut 应用。

ABSTRACT

Although deep models have greatly improved the accuracy and robustness of image segmentation, obtaining segmentation results with highly accurate boundaries and fine structures is still a challenging problem. In this paper, we propose a simple yet powerful Boundary-Aware Segmentation Network (BASNet), which comprises a predict-refine architecture and a hybrid loss, for highly accurate image segmentation. The predict-refine architecture consists of a densely supervised encoder-decoder network and a residual refinement module, which are respectively used to predict and refine a segmentation probability map. The hybrid loss is a combination of the binary cross entropy, structural similarity and intersection-over-union losses, which guide the network to learn three-level (ie, pixel-, patch- and map- level) hierarchy representations. We evaluate our BASNet on two reverse tasks including salient object segmentation, camouflaged object segmentation, showing that it achieves very competitive performance with sharp segmentation boundaries. Importantly, BASNet runs at over 70 fps on a single GPU which benefits many potential real applications. Based on BASNet, we further developed two (close to) commercial applications: AR COPY & PASTE, in which BASNet is integrated with augmented reality for "COPYING" and "PASTING" real-world objects, and OBJECT CUT, which is a web-based tool for automatic object background removal. Both applications have already drawn huge amount of attention and have important real-world impacts. The code and two applications will be publicly available at: https://github.com/NathanUA/BASNet.

研究动机与目标

• 提高图像分割边界与细小结构的空间精度。
• 提出一种简单而有效的 predict-refine 架构，结合深层编码-解码器与残差细化模块。
• 设计一个融合 BCE、SSIM 和 IoU 的混合损失，以对像素、块和映射层次的表示进行监督。
• 展示在显著对象分割和伪装对象分割方面的出色性能，同时实现实际世界应用。

提出的方法

• 提出 BASNet，具备密集监督的编码-解码预测模块和用于细化粗糙图的残差细化模块。
• 使用结合 BCE、SSIM 与 IoU 的混合损失来监督跨像素、块和映射级别的八个输出。
• 实现深度监督方案，使每个解码阶段和一个额外的细化模块生成分割图。
• 通过三层损失平衡边界保真度与全局区域精度。
• 提供两个接近商业化的应用（AR Copy & Paste 与 Object Cut），基于 BASNet 展示实用性。

实验结果

研究问题

• RQ1 BASNet 是否能够为显著对象和伪装对象实现尖锐、准确的边界？
• RQ2 在边界与区域指标上，与标准的编码-解码器及单一损失方法相比，所提出的 predict-refine 架构与混合损失有何表现？
• RQ3 残差细化模块对最终分割质量和边界精度有何影响？
• RQ4 基于 BASNet 的应用（AR Copy & Paste、Object Cut）在现实世界用例中是否提供实用的实时性能？

主要发现

• BASNet 在六个显著对象分割数据集以及 CAM（COD）数据集上表现具有竞争力，且在边界评估指标上更出色。
• 该模型在单个 GPU 上的处理速度超过 70 帧/秒，支持实时或近实时应用。
• 在像素、块和映射级目标上的混合损失（BCE+SSIM+IoU）有效强调了精确边界和细小结构。
• 密集监督的编码-解码预测模块与残差细化模块的组合明显提升了相对于基线的分割质量。
• 两个基于 BASNet 的应用，AR Copy & Paste 与 Object Cut，展示了实际部署潜力和对用户的影响。