[Paper Review] Adaptive Graph Convolutional Network with Attention Graph Clustering for Co-saliency Detection

Kaihua Zhang, Tengpeng Li+4 more|arXiv (Cornell University)|Mar 13, 2020

Visual Attention and Saliency Detection71 references8 citations

TL;DR

This paper proposes GCAGC, an end-to-end adaptive graph convolutional network with attention graph clustering for co-saliency detection. It models long-range intra- and inter-image dependencies via adaptive graph learning and refines predictions using unsupervised attention-based clustering, achieving state-of-the-art performance on iCoseg, Cosal2015, and COCO-SEG with F-scores of 0.8428 and 0.7092, respectively.

ABSTRACT

Co-saliency detection aims to discover the common and salient foregrounds from a group of relevant images. For this task, we present a novel adaptive graph convolutional network with attention graph clustering (GCAGC). Three major contributions have been made, and are experimentally shown to have substantial practical merits. First, we propose a graph convolutional network design to extract information cues to characterize the intra- and interimage correspondence. Second, we develop an attention graph clustering algorithm to discriminate the common objects from all the salient foreground objects in an unsupervised fashion. Third, we present a unified framework with encoder-decoder structure to jointly train and optimize the graph convolutional network, attention graph cluster, and co-saliency detection decoder in an end-to-end manner. We evaluate our proposed GCAGC method on three cosaliency detection benchmark datasets (iCoseg, Cosal2015 and COCO-SEG). Our GCAGC method obtains significant improvements over the state-of-the-arts on most of them.

Motivation & Objective

• Address the challenge of detecting common salient objects across image groups without prior knowledge of their semantic categories.
• Overcome limitations of CNNs in modeling long-range and inter-image dependencies for co-saliency detection.
• Develop an unsupervised method to distinguish common salient objects from other salient foregrounds in a joint optimization framework.
• Design an end-to-end trainable framework that integrates graph learning, clustering, and detection for improved performance.

Proposed method

• Employ a CNN encoder to extract multi-scale features and construct dense feature node graphs from image groups.
• Design an adaptive graph convolutional network (AGCN) that computes direct long-range interactions between any two image positions, capturing intra- and inter-image correspondences.
• Introduce an attention graph clustering module (AGCM) that generates co-attention maps to refine predictions by distinguishing co-salient objects from other salient regions in an unsupervised manner.
• Use a CNN decoder to produce final co-saliency maps, with all components jointly optimized in an end-to-end encoder-decoder framework.
• Implement adaptive graph learning in AGCN to tailor the graph structure to the co-saliency detection task, avoiding fixed graph designs.
• Leverage co-attention maps from AGCM to guide feature refinement and improve spatial consistency and noise suppression.

Experimental results

Research questions

• RQ1 Can adaptive graph convolutional networks effectively model long-range and inter-image dependencies for co-saliency detection?
• RQ2 Can unsupervised attention-based graph clustering improve the discrimination of common salient objects from other salient foregrounds?
• RQ3 Does end-to-end joint optimization of graph learning, clustering, and detection yield superior performance compared to stage-wise or separate training?
• RQ4 How does the proposed method perform under complex scenarios with cluttered backgrounds and multiple interfering objects?

Key findings

• GCAGC achieves the highest F-score of 0.8428 on the Cosal2015 benchmark, outperforming the second-best method (CSMG) by 3.31% and RCGS by 2.78%.
• On COCO-SEG, GCAGC achieves an F-score of 0.7092, surpassing RCGS (0.6814) by 2.78% and CSMG by 3.31%.
• Ablative studies show that removing AGCN leads to a 2.72% drop in Fβ on Cosal2015, demonstrating its critical role in capturing long-range dependencies.
• The AGCM contributes significantly, with a 4.09% drop in Fβ on iCoseg when removed, confirming its effectiveness in refining co-salient regions.
• The adaptive graph learning strategy in AGCN improves performance over fixed graph designs, with a notable 2.22% drop in AP when disabled.
• Qualitative results show that GCAGC produces more spatially coherent co-saliency maps with reduced background noise, even under complex clutter and interference.