[论文解读] CamoFormer: Masked Separable Attention for Camouflaged Object Detection

Bowen Yin, Xuying Zhang+4 more|arXiv (Cornell University)|Dec 10, 2022

Visual Attention and Saliency Detection被引用 27

一句话总结

CamoFormer 在编码器–解码器 Transformer 框架中引入 Masked Separable Attention (MSA)，以分别建模前景和背景线索，在主要 COD 基准数据集上实现了最先进的伪装物体检测。

ABSTRACT

How to identify and segment camouflaged objects from the background is challenging. Inspired by the multi-head self-attention in Transformers, we present a simple masked separable attention (MSA) for camouflaged object detection. We first separate the multi-head self-attention into three parts, which are responsible for distinguishing the camouflaged objects from the background using different mask strategies. Furthermore, we propose to capture high-resolution semantic representations progressively based on a simple top-down decoder with the proposed MSA to attain precise segmentation results. These structures plus a backbone encoder form a new model, dubbed CamoFormer. Extensive experiments show that CamoFormer surpasses all existing state-of-the-art methods on three widely-used camouflaged object detection benchmarks. There are on average around 5% relative improvements over previous methods in terms of S-measure and weighted F-measure.

研究动机与目标

• Motivate and address the challenge of segmenting camouflaged objects that are highly similar to their surroundings.
• Propose a novel Masked Separable Attention (MSA) that splits attention heads to handle foreground, background, and global relations.
• Leverage a progressive top-down decoder to refine segmentation maps using MSA at multiple feature levels.
• Demonstrate state-of-the-art performance on three COD benchmarks and analyze the contributions of MSA components.

提出的方法

• Adopt an encoder–decoder architecture with a Transformer-based backbone (PVTv2) to extract multi-scale features.
• Introduce Masked Separable Attention (MSA) that groups attention heads into Foreground-TA (F-TA), Background-TA (B-TA), and normal TA, each using predicted foreground masks to compute masked queries/keys.
• Generate foreground masks from intermediate predictions and use them to drive F-TA and B-TA computations, while concatenating with normal TA outputs (Z) and passing through a 3x3 conv to 128 channels.
• Progressively fuse features from top to bottom in the decoder by computing an element-wise product before summing with the corresponding encoder features (D_i = up(MSA(D_{i+1})) * E_i + E_i).
• Supervise multi-stage predictions with BCE and IoU losses, summing losses across stages to train the model end-to-end.

实验结果

研究问题

• RQ1 Can foreground-background separated attention improve the segmentation of camouflaged objects compared to standard self-attention in COD?
• RQ2 Does progressive refinement with MSA at multiple decoder levels yield higher quality CAMO masks and borders?
• RQ3 How do the individual MSA components (F-TA, B-TA, and TA) contribute to COD performance?
• RQ4 What are the trade-offs in decoder width (C_d) and how do they affect accuracy vs. computation?

主要发现

• CamoFormer achieves new state-of-the-art results on NC4K, COD10K, and CAMO COD benchmarks.
• On COD10K-test, it reaches 0.786 weighted F-measure and 0.023 MAE (surpassing the second-best FDNet with 0.731 and 0.030).
• MSA provides consistent improvements over baselines and standalone TA, with all three branches (F-TA, B-TA, TA) contributing to performance gains.
• Progressive refinement with MSA yields larger performance gaps in later decoder levels, indicating strong complementarity between MSA and the top-down fusion.
• Ablation shows that increasing decoder channels (C_d) improves performance up to a point (128 channels), balancing accuracy and computation.
• Qualitative visualizations show sharper borders and more complete camouflaged object segmentation compared to prior methods.