[Paper Review] RelationNet2: Deep Comparison Columns for Few-Shot Learning

Xueting Zhang, Yuting Qiang+3 more|arXiv (Cornell University)|Nov 17, 2018

Domain Adaptation and Few-Shot Learning48 references18 citations

TL;DR

RelationNet2 proposes a Deep Comparison Network (DCN) that enhances few-shot learning by jointly learning multiple non-linear distance metrics across hierarchical feature levels via relation modules, while using distributional feature embeddings with Gaussian noise regularization to prevent overfitting. The method achieves state-of-the-art performance on miniImageNet and tiered-ImageNet benchmarks with improved generalization and architectural simplicity.

ABSTRACT

Few-shot deep learning is a topical challenge area for scaling visual recognition to open ended growth of unseen new classes with limited labeled examples. A promising approach is based on metric learning, which trains a deep embedding to support image similarity matching. Our insight is that effective general purpose matching requires non-linear comparison of features at multiple abstraction levels. We thus propose a new deep comparison network comprised of embedding and relation modules that learn multiple non-linear distance metrics based on different levels of features simultaneously. Furthermore, to reduce over-fitting and enable the use of deeper embeddings, we represent images as distributions rather than vectors via learning parameterized Gaussian noise regularization. The resulting network achieves excellent performance on both miniImageNet and tieredImageNet.

Motivation & Objective

• To address the challenge of few-shot visual recognition with limited labeled examples, especially in open-ended learning scenarios.
• To improve generalization in few-shot learning by enabling non-linear, multi-level feature comparison beyond simple linear metrics.
• To reduce overfitting in deep networks when training on few-shot data, particularly with deeper architectures.
• To eliminate the need for task-specific fine-tuning while maintaining fast inference and training.
• To enable effective gradient flow and representation learning across multiple abstraction levels in feature hierarchies.

Proposed method

• The framework uses a deep embedding network that outputs feature distributions via parameterized Gaussian noise regularization, replacing standard vector embeddings.
• It introduces multiple relation modules operating at different levels of the feature hierarchy, each learning a non-linear distance metric on corresponding feature representations.
• Each relation module is deeply supervised, enabling better gradient flow and improved training stability across the hierarchy.
• The method employs module weighting via learnable attention to dynamically combine predictions from different relation modules, reducing reliance on manual hyperparameter tuning.
• The network is trained end-to-end using auxiliary few-shot tasks, learning task-agnostic meta-knowledge for fast adaptation to new classes.
• The architecture supports both classification and similarity matching, with inference based on nearest-neighbor comparison in the embedding space.

Experimental results

Research questions

• RQ1 Can multiple non-linear comparison modules operating at different abstraction levels improve few-shot generalization compared to single-level metrics?
• RQ2 Does representing features as distributions via stochastic regularization enhance generalization in few-shot learning without increasing inference cost?
• RQ3 Can deep supervision of relation modules improve performance and training stability in deep metric learning frameworks?
• RQ4 How do relation modules at different depths of the feature hierarchy contribute to prediction diversity and complementarity?
• RQ5 To what extent does the proposed method outperform existing state-of-the-art few-shot learning models on challenging benchmarks like tiered-ImageNet?

Key findings

• RelationNet2 achieves state-of-the-art performance on both miniImageNet and tiered-ImageNet benchmarks, outperforming existing methods including RelationNet, ProtoNet, and MAML.
• The ablation study confirms that deep supervision of relation modules significantly improves performance, with DCN-No Deep Sup. showing a notable drop in accuracy.
• Learning attention weights for module combination improves performance over fixed manual weights and eliminates the need for hyperparameter tuning.
• Each individual relation module performs competitively, but their combination yields the best results, confirming the value of multi-level feature comparison.
• The relation modules show low prediction correlation (down to 0.34), indicating diverse and non-redundant predictions, with adjacent modules showing higher correlation than non-adjacent ones.
• The analysis of score-distance correlation confirms that relation module scores align with semantic distances in ImageNet, validating their interpretability and discriminative power.