[论文解读] MSANet: Multi-Similarity and Attention Guidance for Boosting Few-Shot Segmentation
MSANet 引入一个多相似性模块和一个注意力模块,通过利用多层视觉对应关系和定向注意力,提升少样本分割,在标准基准上实现最先进结果。
Few-shot segmentation aims to segment unseen-class objects given only a handful of densely labeled samples. Prototype learning, where the support feature yields a singleor several prototypes by averaging global and local object information, has been widely used in FSS. However, utilizing only prototype vectors may be insufficient to represent the features for all training data. To extract abundant features and make more precise predictions, we propose a Multi-Similarity and Attention Network (MSANet) including two novel modules, a multi-similarity module and an attention module. The multi-similarity module exploits multiple feature-maps of support images and query images to estimate accurate semantic relationships. The attention module instructs the network to concentrate on class-relevant information. The network is tested on standard FSS datasets, PASCAL-5i 1-shot, PASCAL-5i 5-shot, COCO-20i 1-shot, and COCO-20i 5-shot. The MSANet with the backbone of ResNet-101 achieves the state-of-the-art performance for all 4-benchmark datasets with mean intersection over union (mIoU) of 69.13%, 73.99%, 51.09%, 56.80%, respectively. Code is available at https://github.com/AIVResearch/MSANet
研究动机与目标
- Motivate improving few-shot segmentation beyond prototype-based guidance by incorporating richer visual correspondences.
- Leverage multi-layer feature correlations to capture diverse contextual information between support and query images.
- Incorporate an efficient attention mechanism to focus on class-relevant regions using few support examples.
- Integrate the proposed modules with a base/ensemble learner framework to refine segmentation results.
- Demonstrate state-of-the-art performance on standard FSS benchmarks across multiple backbones.
提出的方法
- Introduce two guiding modules: a multi-layer similarity module for dense visual correspondence across support and query images, and a lightweight attention module to emphasize target-class content.
- Compute cosine similarity across multi-layer backbone features (blocks 2–4) to form cross-image correspondences, followed by 1x1 convolutions to fuse features.
- Mask and squeeze support features with the corresponding masks to reduce background noise and form robust visual correspondence maps.
- Generate an attention feature-map from concatenated intermediate support/query features through a simple 1x1 convolutional block and pooling-based attention pipeline.
- Fuse multi-layer similarity maps, attention maps, prior masks, and class prototype vectors in an ASPP-based feature enrichment module, then predict a binary segmentation mask via a small classifier head.
- Enhance final predictions with a base learner (PSPNet-like) trained on base classes and an ensemble module for refinement.
- Train with binary cross-entropy loss over episodes, following standard meta-learning protocols, with optional averaging for K-shot settings.
实验结果
研究问题
- RQ1Can using multi-layer visual correspondences between support and query images improve segmentation accuracy in few-shot settings?
- RQ2Does a lightweight attention module guided by limited support data help suppress base-class biases and better isolate novel-class objects?
- RQ3What is the impact of integrating multi-layer similarity and attention with a base learner/ensemble on overall FSS performance?
- RQ4How do different backbone layers contribute to the quality of visual correspondence for FSS?
- RQ5Is MSANet scalable to 1-shot and 5-shot settings on common FSS benchmarks (PASCAL-5i, COCO-20i) with various backbones?
主要发现
- MSANet achieves state-of-the-art results on four FSS benchmarks under 1-shot and 5-shot settings (Pascal-5i and COCO-20i).
- With ResNet101 backbone, MSANet attains mIoU of 69.13% (Pascal-5i 1-shot) and 73.99% (Pascal-5i 5-shot), and 51.09% (COCO-20i 1-shot) and 56.80% (COCO-20i 5-shot).
- Ablation studies show the multi-layer similarity and attention modules provide the largest gains, outperforming prototype-only baselines and approach new SOTA levels.
- Combining the two modules with prototype-based elements (prior mask, class prototype vector) and the base-learner ensemble yields the best final performance.
- Visualization indicates that combining low-, mid-, and high-level features for correspondence yields richer guidance than any single-layer approach.
- MSANet demonstrates robustness across backbone choices (VGG16, ResNet50, ResNet101) and shot settings.
阅读本论文
在 Nubint 阅读器中打开原文,边读边就任意段落提问。
在 Nubint 中阅读本论文本解读由 AI 根据论文原文生成,引用前请核对原文。