[Paper Review] Very Deep Convolutional Networks for Large-Scale Image Recognition

Karen Simonyan, Andrew Zisserman|arXiv (Cornell University)|Sep 4, 2014

Advanced Vision and Imaging30 references75,407 citations

TL;DR

The paper investigates the impact of very deep ConvNets (11–19 weight layers) built with 3x3 filters on large-scale image recognition, achieving state-of-the-art results and releasing top models.

ABSTRACT

In this work we investigate the effect of the convolutional network depth on its accuracy in the large-scale image recognition setting. Our main contribution is a thorough evaluation of networks of increasing depth using an architecture with very small (3x3) convolution filters, which shows that a significant improvement on the prior-art configurations can be achieved by pushing the depth to 16-19 weight layers. These findings were the basis of our ImageNet Challenge 2014 submission, where our team secured the first and the second places in the localisation and classification tracks respectively. We also show that our representations generalise well to other datasets, where they achieve state-of-the-art results. We have made our two best-performing ConvNet models publicly available to facilitate further research on the use of deep visual representations in computer vision.

Motivation & Objective

• Evaluate how network depth affects accuracy in large-scale image recognition.
• Demonstrate that small 3x3 convolution filters can build highly accurate deep networks.
• Assess training/testing strategies (scale variation, multi-scale training, multi-crop evaluation) and generalization to other datasets.
• Release best-performing models to facilitate further research.

Proposed method

• Use a fixed input size of 224x224 with 3x3 convolutional filters and 1-pixel padding to preserve resolution.
• Stack multiple conv layers (A–E) with increasing depth from 11 to 19 weight layers, using small widths that grow after each max-pooling layer.
• Apply ReLU activations and avoid LRN in deeper networks to reduce memory and computation while maintain performance.
• Train with SGD (batch 256, momentum 0.9), L2 weight decay, and dropout on the first two fully-connected layers; employ scale-based data augmentation and multi-scale training.
• Convert fully-connected layers to convolutional layers during testing to enable dense evaluation over whole images and perform sum-pooling of class scores; evaluate with single-scale, multi-scale, and multi-crop strategies; ensemble multiple models for improved accuracy.

Experimental results

Research questions

• RQ1 How does increasing depth (11 to 19 weight layers) affect top-1 and top-5 accuracy on ILSVRC-2012/2014?
• RQ2 Do 3x3 filter stacks offer advantages over larger filters when depth increases?
• RQ3 What is the impact of training and testing with multi-scale and multi-crop evaluation on final accuracy?
• RQ4 Can the learned very deep representations generalize to other datasets beyond ImageNet?
• RQ5 What is the performance of single models versus ensembles for this architecture?

Key findings

• Deeper networks (up to 19 weight layers) improve performance compared to shallower ones.
• 3x3 filter stacks with ReLU outperform equivalent or larger-filter architectures; adding 1x1 convolutions provides non-linearity but is not superior to deeper 3x3 stacks without losing spatial context.
• Scale jittering during training and testing (multi-scale inputs) yields significant accuracy gains over fixed-scale training.
• Dense evaluation and multi-crop evaluation are complementary; model ensembles further reduce error, achieving 7.0% top-5 test error for the best single model and 6.8% with ensemble and multi-crop.
• The approach secured 2nd place (7.3% top-5 test error) in ILSVRC-2014 classification and outperformed several prior architectures, approaching GoogLeNet performance with a simpler, deeper design.
• The paper released its two best-performing models to facilitate further research.