[Paper Review] Weakly Supervised One-Shot Detection with Attention Siamese Networks

Gil Keren, Maximilian Schmitt+2 more|arXiv (Cornell University)|Jan 10, 2018

Anomaly Detection Techniques and Applications35 references10 citations

TL;DR

This paper proposes a weakly supervised one-shot detection model using an attention-enhanced Siamese network that identifies and localizes novel classes from a single exemplar and a target image, outperforming baseline methods on vision and audio datasets by leveraging class-agnostic feature learning and attention-based feature refinement.

ABSTRACT

Neural network models that are not conditioned on class identities were shown to facilitate knowledge transfer between classes and to be well-suited for one-shot learning tasks. Following this motivation, we further explore and establish such models and present a novel neural network architecture for the task of weakly supervised one-shot detection. Our model is only conditioned on a single exemplar of an unseen class and a larger target example that may or may not contain an instance of the same class as the exemplar. By pairing a Siamese similarity network with an attention mechanism, we design a model that manages to simultaneously identify and localise instances of classes unseen at training time. In experiments with datasets from the computer vision and audio domains, the proposed method considerably outperforms the baseline methods for the weakly supervised one-shot detection task.

Motivation & Objective

• To address the challenge of detecting novel object classes in images or audio with minimal supervision.
• To enable one-shot detection without requiring class-specific annotations during training.
• To improve zero-shot generalization by leveraging class-agnostic feature learning.
• To design a model that simultaneously localizes and identifies unseen classes using only a single exemplar.

Proposed method

• The model employs a Siamese network to compute similarity between a single exemplar image and regions of a target image.
• An attention mechanism is integrated to dynamically highlight relevant features in the target image based on the exemplar.
• The Siamese branch learns class-agnostic representations to facilitate knowledge transfer across classes.
• The network is trained end-to-end using weak supervision, where only the presence or absence of the exemplar class in the target image is provided.
• Feature maps from the Siamese network are fused with attention weights to refine localization predictions.
• The architecture enables joint detection and localization of unseen classes without requiring bounding box annotations during training.

Experimental results

Research questions

• RQ1 Can a class-agnostic neural network architecture effectively detect and localize novel classes in one-shot settings with minimal supervision?
• RQ2 How does integrating an attention mechanism improve localization accuracy in weakly supervised one-shot detection?
• RQ3 To what extent can knowledge transfer between classes be enhanced by using Siamese networks with shared weights and attention?
• RQ4 Does the proposed method generalize across different domains, such as computer vision and audio?

Key findings

• The proposed method significantly outperforms baseline methods on weakly supervised one-shot detection benchmarks.
• The integration of attention with Siamese networks improves localization accuracy by focusing on relevant image regions matching the exemplar.
• The model generalizes well to unseen classes due to its class-agnostic feature learning strategy.
• The approach achieves strong performance on both image and audio datasets, demonstrating cross-domain applicability.