[Paper Review] Caffe: Convolutional Architecture for Fast Feature Embedding

Yangqing Jia, Evan Shelhamer+6 more|arXiv (Cornell University)|Jun 20, 2014

Advanced Image and Video Retrieval Techniques7 references4,292 citations

TL;DR

Caffe is an open-source, BSD-licensed C++ framework with Python and MATLAB bindings for fast training, deployment, and experimentation with convolutional neural networks, featuring modular layers, GPU acceleration, and pretrained models.

ABSTRACT

Caffe provides multimedia scientists and practitioners with a clean and modifiable framework for state-of-the-art deep learning algorithms and a collection of reference models. The framework is a BSD-licensed C++ library with Python and MATLAB bindings for training and deploying general-purpose convolutional neural networks and other deep models efficiently on commodity architectures. Caffe fits industry and internet-scale media needs by CUDA GPU computation, processing over 40 million images a day on a single K40 or Titan GPU ($\approx$ 2.5 ms per image). By separating model representation from actual implementation, Caffe allows experimentation and seamless switching among platforms for ease of development and deployment from prototyping machines to cloud environments. Caffe is maintained and developed by the Berkeley Vision and Learning Center (BVLC) with the help of an active community of contributors on GitHub. It powers ongoing research projects, large-scale industrial applications, and startup prototypes in vision, speech, and multimedia.

Motivation & Objective

• Provide a clean, modifiable framework for state-of-the-art deep learning algorithms.
• Enable rapid research and deployment from prototyping to cloud environments.
• Offer end-to-end training, testing, finetuning, and deployment capabilities.
• Include pre-trained reference models to encourage reproducible research.

Proposed method

• Modular architecture with a complete set of layer types (convolution, pooling, nonlinearities, losses).
• Separation of network representation from implementation via Protocol Buffers configuration files.
• 4D blob data storage with on-demand host/device memory management to unify CPU/GPU work.
• Single CPU/GPU switch to run networks without changing definitions.
• Training via stochastic gradient descent with learning rate schedules, momentum, and snapshots.
• Finetuning existing models by transferring weights to new architectures or data.

Experimental results

Research questions

• RQ1 How can we design a deep learning framework that is both fast (GPU-accelerated) and easily adaptable for researchers and industry.
• RQ2 How can network architectures be specified and deployed independently of their implementation?
• RQ3 Can pretrained reference models accelerate research and enable reproducible experiments?
• RQ4 What mechanisms are needed to support seamless CPU/GPU deployment and easy finetuning?
• RQ5 How can data, models, and experiments be organized to scale from prototyping to production?

Key findings

• Caffe achieves fast GPU computation suitable for large-scale media workloads (over 40 million images per day on a single K40 or Titan GPU).
• Networks are defined in config files (Protocol Buffers) and can run on CPU or GPU with identical results.
• Every module has tests, promoting experimental rigor and reliability.
• Pre-trained reference models are provided for rapid experimentation and reproduction of results.
• The framework supports end-to-end training, testing, finetuning, and deployment with Python and MATLAB bindings.
• Caffe emphasizes separation of representation from implementation, enabling easy switching across platforms and deployment environments.