[Paper Review] Impostor Networks for Fast Fine-Grained Recognition

Vadim Lebedev, Artem Babenko+1 more|arXiv (Cornell University)|Jun 13, 2018

Advanced Neural Network Applications25 references3 citations

TL;DR

This paper introduces impostor networks, a lightweight deep learning architecture that combines a compact convolutional neural network (e.g., SqueezeNet) with a learnable, non-parametric radial basis function (RBF) classifier to enable high-accuracy fine-grained image recognition on CPU-only and low-power platforms. By jointly training the feature extractor and RBF classifier—where the RBF centers (impostors) are not fixed to training samples but evolve during training—the method achieves significant accuracy gains over baseline networks with minimal computational overhead.

ABSTRACT

In this work we introduce impostor networks, an architecture that allows to perform fine-grained recognition with high accuracy and using a light-weight convolutional network, making it particularly suitable for fine-grained applications on low-power and non-GPU enabled platforms. Impostor networks compensate for the lightness of its `backend' network by combining it with a lightweight non-parametric classifier. The combination of a convolutional network and such non-parametric classifier is trained in an end-to-end fashion. Similarly to convolutional neural networks, impostor networks can fit large-scale training datasets very well, while also being able to generalize to new data points. At the same time, the bulk of computations within impostor networks happen through nearest neighbor search in high-dimensions. Such search can be performed efficiently on a variety of architectures including standard CPUs, where deep convolutional networks are inefficient. In a series of experiments with three fine-grained datasets, we show that impostor networks are able to boost the classification accuracy of a moderate-sized convolutional network considerably at a very small computational cost.

Motivation & Objective

• Address the challenge of deploying high-accuracy fine-grained image classifiers on low-power, non-GPU platforms such as mobile devices.
• Reduce the computational and model size burden of state-of-the-art deep networks while maintaining or improving classification accuracy.
• Explore the use of a learnable, non-parametric classifier (RBF) that can compensate for the limited representational capacity of lightweight CNNs.
• Improve generalization and open-set detection capabilities by allowing RBF centers (impostors) to drift from training examples during training.
• Demonstrate that joint end-to-end training of the CNN and RBF classifier enables strong performance even with small, efficient networks.

Proposed method

• Propose a hybrid architecture where a lightweight convolutional network (e.g., SqueezeNet) is followed by a learnable RBF classifier with non-fixed centers.
• Train the entire system end-to-end using a loss function that includes both cross-entropy and a neighborhood-component analysis (NCA)-like term to encourage class compactness.
• Allow the RBF centers (impostors) to be learnable parameters that are not tied to any specific training sample, enabling them to drift during optimization.
• Use high-dimensional CNN features as input to the RBF classifier, which performs classification via weighted voting based on L2 distances to impostor centers.
• Implement three training variants: fixed impostors (initialized to training features), loose impostors (learnable and updated), and a hybrid approach.
• Apply standard model compression techniques (e.g., pruning, quantization) to reduce memory overhead without significant accuracy drop.

Experimental results

Research questions

• RQ1 Can a lightweight CNN combined with a learnable non-parametric classifier achieve high fine-grained recognition accuracy on low-power hardware?
• RQ2 Does allowing RBF centers (impostors) to evolve during training—rather than fixing them to training samples—improve generalization and accuracy?
• RQ3 How does the impostor network perform in open-set recognition scenarios, where unknown classes are present at test time?
• RQ4 What is the computational and memory overhead of the impostor network compared to standard CNNs, and can it be reduced via compression?
• RQ5 To what extent can the performance of a small network like SqueezeNet be boosted using this method on fine-grained datasets?

Key findings

• Impostor networks significantly boost the classification accuracy of a moderate-sized CNN (e.g., SqueezeNet) on fine-grained datasets such as CUB-200-2011 and Stanford Cars, with minimal computational cost.
• The loose impostor variant, where RBF centers are trained end-to-end, achieves the highest accuracy, demonstrating that learned, non-fixed impostors improve generalization.
• On the CUB-200-2011 dataset, the impostor network with SqueezeNet outperforms the standard cross-entropy baseline by a large margin, even surpassing some deeper models in accuracy while being much faster.
• The method improves open-set recognition: impostor networks show higher entropy (lower confidence) on unseen classes (e.g., non-bird images), indicating better uncertainty estimation than standard CNNs.
• The memory overhead of impostor networks is minimal and can be further reduced via model compression (e.g., pruning, quantization) without significant accuracy loss.
• The approach enables efficient inference on CPUs, making it suitable for mobile and edge deployment, unlike deep models that require GPU acceleration.