[Paper Review] Deep Image: Scaling up Image Recognition

Wu Ren, Shengen Yan+3 more|arXiv (Cornell University)|Jan 13, 2015

Advanced Neural Network Applications54 references330 citations

TL;DR

Deep Image proposes a scalable image recognition system using a custom supercomputer, optimized parallel algorithms, larger deep neural networks, advanced data augmentation, and multi-scale high-resolution inputs. It achieves state-of-the-art performance on multiple computer vision benchmarks through end-to-end deep learning optimization.

ABSTRACT

We present a state-of-the-art image recognition system, Deep Image, developed using end-to-end deep learning. The key components are a custom-built supercomputer dedicated to deep learning, a highly optimized parallel algorithm using new strategies for data partitioning and communication, larger deep neural network models, novel data augmentation approaches, and usage of multi-scale high-resolution images. Our method achieves excellent results on multiple challenging computer vision benchmarks.

Motivation & Objective

• To develop a scalable image recognition system capable of handling large-scale visual datasets with high accuracy.
• To address the limitations of existing deep learning systems in training large models efficiently on high-resolution images.
• To improve recognition accuracy by leveraging novel data augmentation and multi-scale input processing.
• To optimize communication and data partitioning in distributed training for faster convergence and scalability.

Proposed method

• Utilizes a custom-built supercomputer specifically designed for accelerating deep learning workloads.
• Employs a highly optimized parallel algorithm with new strategies for data partitioning and inter-node communication.
• Trains larger deep neural network architectures than previously feasible due to system scalability.
• Applies novel data augmentation techniques to improve model generalization and robustness.
• Processes images at multiple scales during training and inference to enhance feature learning.
• Uses end-to-end deep learning to jointly optimize all components of the recognition pipeline.

Experimental results

Research questions

• RQ1 How can deep learning systems be scaled to train larger models efficiently on high-resolution images?
• RQ2 What communication and data partitioning strategies maximize training throughput in distributed deep learning?
• RQ3 To what extent do multi-scale inputs and advanced data augmentation improve recognition accuracy?
• RQ4 Can a custom supercomputer architecture outperform standard GPU clusters in large-scale image recognition tasks?

Key findings

• Deep Image achieves state-of-the-art performance on multiple challenging computer vision benchmarks.
• The custom supercomputer enables faster training of larger deep neural networks than conventional systems.
• Optimized data partitioning and communication reduce training time and improve scalability.
• Multi-scale high-resolution inputs significantly enhance feature representation and recognition accuracy.
• Novel data augmentation strategies improve model generalization without increasing inference cost.