[Paper Review] Refining Architectures of Deep Convolutional Neural Networks

Sukrit Shankar, Duncan Robertson+3 more|arXiv (Cornell University)|Apr 22, 2016

Advanced Neural Network Applications23 references4 citations

TL;DR

This paper proposes a novel architecture refinement method for deep convolutional neural networks (CNNs) that enhances accuracy while reducing model size through two operations: layer stretching (increasing hidden units) and symmetrical splitting (grouping input/output channels). Applied to GoogleNet and VGG-11 on SUN Attributes and CAMIT-NSAD datasets, the method achieves higher precision than baselines—especially on SUN Attributes—while reducing parameters, demonstrating a win-win trade-off in some cases and a viable alternative in others.

ABSTRACT

Deep Convolutional Neural Networks (CNNs) have recently evinced immense success for various image recognition tasks. However, a question of paramount importance is somewhat unanswered in deep learning research - is the selected CNN optimal for the dataset in terms of accuracy and model size? In this paper, we intend to answer this question and introduce a novel strategy that alters the architecture of a given CNN for a specified dataset, to potentially enhance the original accuracy while possibly reducing the model size. We use two operations for architecture refinement, viz. stretching and symmetrical splitting. Our procedure starts with a pre-trained CNN for a given dataset, and optimally decides the stretch and split factors across the network to refine the architecture. We empirically demonstrate the necessity of the two operations. We evaluate our approach on two natural scenes attributes datasets, SUN Attributes and CAMIT-NSAD, with architectures of GoogleNet and VGG-11, that are quite contrasting in their construction. We justify our choice of datasets, and show that they are interestingly distinct from each other, and together pose a challenge to our architectural refinement algorithm. Our results substantiate the usefulness of the proposed method.

Motivation & Objective

• To address the open question of whether a pre-trained CNN architecture is optimal for a given dataset in terms of accuracy and model size.
• To develop a strategy that refines CNN architecture by modifying layer width and connectivity patterns to improve performance.
• To provide a practical alternative for users who want to enhance accuracy or reduce model size without retraining from scratch.
• To evaluate the effectiveness of architectural refinement on diverse, challenging datasets with contrasting CNN architectures.

Proposed method

• The method starts with a pre-trained CNN and applies two operations: stretching (increasing the number of hidden units in a layer) and symmetrical splitting (dividing input and output channels into K equal groups and connecting corresponding groups).
• Stretching is applied to increase representational capacity in layers where class separation is suboptimal, while symmetrical splitting reduces parameter count by introducing grouped convolutions.
• The algorithm greedily selects optimal stretch and split factors for each layer based on class separation performance at each convolutional layer, using a validation set to guide decisions.
• The approach is applied only to convolutional layers, preserving the original network depth and overall structure.
• The method is evaluated using precision@k as the metric, with k set to the maximum number of positive labels per image in the test set.
• Training uses Caffe with adaptive learning rate scheduling and 100 epochs; refinement is performed post-pretraining without retraining the full network.

Experimental results

Research questions

• RQ1 Can architectural refinement through stretching and symmetrical splitting improve the accuracy of a pre-trained CNN on a given dataset without increasing model size?
• RQ2 How does the combination of stretching and splitting compare to using either operation in isolation for performance enhancement?
• RQ3 Does the proposed method achieve better trade-offs between accuracy and model size than existing sparsity-based baselines?
• RQ4 Why does the method succeed on some datasets (e.g., SUN Attributes) but not others (e.g., CAMIT-NSAD) in improving precision?
• RQ5 Can a greedy, layer-wise refinement strategy effectively optimize feature representation when class-specific information is encoded across multiple layers?

Key findings

• On the SUN Attributes dataset, the proposed method achieved higher precision than both baseline models and sparsity-based approaches, with a significant reduction in model size, demonstrating a win-win outcome.
• For VGG-11 and GoogleNet on SUN Attributes, the method improved precision while reducing the number of parameters, especially in convolutional layers.
• The combination of stretching and symmetrical splitting was more effective than using either operation alone, indicating synergistic benefits in architectural refinement.
• On CAMIT-NSAD, the method reduced model size but slightly decreased precision, though the resulting model still outperformed sparsity-based baselines in accuracy.
• The greedy nature of the algorithm may limit performance on datasets with complex, multi-level feature encoding (e.g., attribute-noun pairs in CAMIT-NSAD), where joint optimization across layers is more critical.
• The results show that architectural refinement is a viable and complementary strategy to sparsification, especially when users prioritize accuracy and model efficiency.