[Paper Review] Cross Attention Network for Few-shot Classification

Ruibing Hou, Hong Chang+3 more|arXiv (Cornell University)|Oct 17, 2019

Domain Adaptation and Few-Shot Learning143 citations

TL;DR

CAN introduces a Cross Attention Module to highlight target objects by modeling semantic relevance between class and query features, plus a transductive inference step to augment support with confident query samples, achieving state-of-the-art results on few-shot benchmarks.

ABSTRACT

Few-shot classification aims to recognize unlabeled samples from unseen classes given only few labeled samples. The unseen classes and low-data problem make few-shot classification very challenging. Many existing approaches extracted features from labeled and unlabeled samples independently, as a result, the features are not discriminative enough. In this work, we propose a novel Cross Attention Network to address the challenging problems in few-shot classification. Firstly, Cross Attention Module is introduced to deal with the problem of unseen classes. The module generates cross attention maps for each pair of class feature and query sample feature so as to highlight the target object regions, making the extracted feature more discriminative. Secondly, a transductive inference algorithm is proposed to alleviate the low-data problem, which iteratively utilizes the unlabeled query set to augment the support set, thereby making the class features more representative. Extensive experiments on two benchmarks show our method is a simple, effective and computationally efficient framework and outperforms the state-of-the-arts.

Motivation & Objective

• Address the instability of discriminative features in few-shot classification due to unseen classes and low data.
• Propose a Cross Attention Module (CAM) to highlight target regions by learning cross correlations between class and query features.
• Introduce a transductive inference algorithm to enrich class representations using unlabeled query samples.
• Demonstrate that CAN is simple, efficient, and achieves state-of-the-art results on standard benchmarks.

Proposed method

• Introduce Cross Attention Module (CAM) that computes cross attention maps between class feature maps and query feature maps via a correlation layer and a meta fusion layer.
• Compute a class correlation map R^p and a query correlation map R^q from cosine similarities of local features, then generate attention maps A^p and A^q through a meta-learner that outputs a kernel w.
• Apply residual attention by weighting features with 1 + A^p and 1 + A^q to obtain discriminative P̄ and Q̄.
• Train CAN with a joint loss L = λL1 + L2, where L1 is a nearest-neighbor based local supervision and L2 is a global classification loss.
• During inference, use inductive or transductive strategies; transductive inference augments the support set with pseudo-labeled query samples to iteratively refine class features.
• Optionally extend CAM with transductive inference to other models (e.g., Matching Network, Prototypical Network, Relation Network).

Experimental results

Research questions

• RQ1 Can cross-attention between support (class) and query features improve discrimination for unseen classes in few-shot tasks?
• RQ2 Does a transductive inference strategy that augments the support set with confidently pseudo-labeled query samples improve performance under low data conditions?
• RQ3 Is the cross attention approach computationally efficient enough to be practical for standard few-shot benchmarks?
• RQ4 Can CAM-guided features enhance both inductive and transductive few-shot classification settings across multiple datasets?

Key findings

• CAN achieves state-of-the-art results on miniImageNet and tieredImageNet in 5-way 1-shot and 5-way 5-shot settings.
• CAN+.T (transductive CAN) outperforms prior transductive methods by up to 8% in 1-shot and 5% in 5-shot on the reported benchmarks.
• Ablation studies show that global classification loss and the cross-attention module substantially improve performance, and the meta-learner in CAM effectively generates adaptive kernels.
• Transductive inference designed for CAN can be generalized to improve other few-shot models (Matching Network, Prototypical Network, Relation Network) as well.
• CAN achieves these gains with modest computational overhead and parameter count, thanks to CAM’s efficient correlation-based attention and a lightweight meta-learner.