[Paper Review] DenseNet: Implementing Efficient ConvNet Descriptor Pyramids

Forrest Iandola, Matthew W. Moskewicz+4 more|arXiv (Cornell University)|Apr 7, 2014

Advanced Neural Network Applications14 references657 citations

TL;DR

DenseNet proposes an efficient system for computing dense, multiscale convolutional neural network (CNN) feature pyramids from pre-trained classifiers, enabling fast and accurate object detection by reusing shared feature computations across overlapping regions. It achieves a 10x speedup over traditional per-region CNN inference, reducing 2000 region proposals from 10 seconds to 1 second on an NVIDIA K20 GPU while maintaining descriptor fidelity.

ABSTRACT

Convolutional Neural Networks (CNNs) can provide accurate object classification. They can be extended to perform object detection by iterating over dense or selected proposed object regions. However, the runtime of such detectors scales as the total number and/or area of regions to examine per image, and training such detectors may be prohibitively slow. However, for some CNN classifier topologies, it is possible to share significant work among overlapping regions to be classified. This paper presents DenseNet, an open source system that computes dense, multiscale features from the convolutional layers of a CNN based object classifier. Future work will involve training efficient object detectors with DenseNet feature descriptors.

Motivation & Objective

• To address the high computational cost of sliding-window object detection using CNNs, which scales poorly with the number of region proposals.
• To enable efficient, dense feature extraction from pre-trained CNNs for multiscale and multi-aspect-ratio region proposals.
• To provide an open-source, interoperable implementation integrated with Caffe for research and development in object detection.
• To validate that densely computed features approximate per-region CNN descriptors accurately, minimizing performance loss.
• To support future training of efficient object detectors using shared, multiscale CNN descriptors.

Proposed method

• DenseNet computes a full feature pyramid for an entire image in one forward pass, avoiding redundant computation across overlapping regions.
• It uses a simplified mean pixel subtraction (instead of per-channel mean image subtraction) to center input features, reducing complexity without significant accuracy loss.
• The system supports non-square region proposals by warping input images to different aspect ratios before feature extraction.
• Feature pyramids are constructed by applying the same CNN to downsampled and warped versions of the input image at multiple scales.
• The implementation is integrated into the Caffe deep learning framework with MATLAB and Python APIs for easy use in detection pipelines.
• It leverages the receptive field and feature hierarchy of pre-trained CNNs (e.g., AlexNet) to extract rich, spatially dense descriptors.

Experimental results

Research questions

• RQ1 Can dense, multiscale CNN feature pyramids be computed efficiently enough to make sliding-window object detection with deep networks practical?
• RQ2 How well do densely computed features approximate per-region CNN descriptors in isolation?
• RQ3 What is the computational overhead of supporting multiple aspect ratios and scale levels in feature extraction?
• RQ4 To what extent can simplified data centering (mean pixel) replace full mean image subtraction without degrading performance?
• RQ5 Can an open-source, Caffe-integrated system enable faster prototyping and comparison of CNN-based object detectors?

Key findings

• DenseNet reduces the time to compute 2000 region proposals from 10 seconds to 1 second on an NVIDIA K20 GPU, achieving a 10x speedup.
• Using a single mean pixel value for centering reduces top-1 accuracy by only 0.2% compared to full mean image subtraction, validating the simplification.
• Visual comparison shows that descriptors extracted from DenseNet’s pre-computed pyramids are visually similar to those computed independently per region, confirming fidelity.
• The system supports multiple aspect ratios by warping input images prior to feature extraction, enabling flexible detection pipeline integration.
• The open-source integration with Caffe and availability of MATLAB/Python APIs facilitate interoperability with existing detection frameworks like DPM and R-CNN.
• The approach enables efficient feature reuse across overlapping regions, making dense CNN-based detection feasible at scale.