[Paper Review] Object Detection with Deep Learning: A Review

Zhong‐Qiu Zhao, Peng Zheng+2 more|arXiv (Cornell University)|Jul 15, 2018

Visual Attention and Saliency Detection173 references82 citations

TL;DR

A comprehensive survey of deep learning-based object detection, tracing from R-CNN to modern region-based and regression/classification frameworks, with analysis across generic and specific tasks.

ABSTRACT

Due to object detection's close relationship with video analysis and image understanding, it has attracted much research attention in recent years. Traditional object detection methods are built on handcrafted features and shallow trainable architectures. Their performance easily stagnates by constructing complex ensembles which combine multiple low-level image features with high-level context from object detectors and scene classifiers. With the rapid development in deep learning, more powerful tools, which are able to learn semantic, high-level, deeper features, are introduced to address the problems existing in traditional architectures. These models behave differently in network architecture, training strategy and optimization function, etc. In this paper, we provide a review on deep learning based object detection frameworks. Our review begins with a brief introduction on the history of deep learning and its representative tool, namely Convolutional Neural Network (CNN). Then we focus on typical generic object detection architectures along with some modifications and useful tricks to improve detection performance further. As distinct specific detection tasks exhibit different characteristics, we also briefly survey several specific tasks, including salient object detection, face detection and pedestrian detection. Experimental analyses are also provided to compare various methods and draw some meaningful conclusions. Finally, several promising directions and tasks are provided to serve as guidelines for future work in both object detection and relevant neural network based learning systems.

Motivation & Objective

• Survey the history and foundations of deep learning and CNNs relevant to object detection.
• Summarize generic object detection architectures and their evolution from region proposal-based to regression/classification frameworks.
• Review applications in salient object detection, face detection, and pedestrian detection and compare performance insights.
• Highlight limitations of early methods and present subsequent improvements and practical considerations.

Proposed method

• Review and categorize object detection frameworks into region-proposal based and regression/classification-based pipelines.
• Discuss key architectures (R-CNN, SPP-net, Fast R-CNN, Faster R-CNN, R-FCN, FPN, Mask R-CNN) and their training strategies.
• Explain the Region Proposal Network (RPN) and its role in end-to-end training and speed gains.
• Describe improvements for efficiency and accuracy, including RoI pooling/align, multi-task losses, and feature pyramids (FPN).
• Summarize specific-task adaptations for salient, face, and pedestrian detection.

Experimental results

Research questions

• RQ1 What are the main architectural paradigms in deep learning-based object detection and how have they evolved over time?
• RQ2 How do region-proposal-based methods compare to regression/classification-based methods in terms of accuracy and efficiency?
• RQ3 What are the key techniques that bridged the gap to real-time, end-to-end object detection?
• RQ4 How do specialized tasks (salient/object, face, pedestrian) influence detector design and performance?

Key findings

• R-CNN achieved a mean average precision (mAP) of 53.3% on PASCAL VOC 2012, marking a significant improvement over prior methods.
• SPP-net and Fast R-CNN improved efficiency and accuracy by reusing convolutional feature maps and introducing RoI pooling with multi-task loss.
• Faster R-CNN introduced Region Proposal Networks (RPN) to generate proposals end-to-end, enabling faster and more integrated training with real-time-like performance (e.g., 5 FPS on GPUs on VOC07/12).
• R-FCN proposed a fully convolutional architecture with position-sensitive RoI pooling to balance translation variance and efficiency, achieving strong results on PASCAL VOC and COCO.
• FPN introduced a top-down, lateral connection-based feature pyramid to unify multi-scale feature representation, enhancing detection across scales.
• Mask R-CNN extended Faster R-CNN with an independent segmentation branch and RoIAlign to improve mask accuracy without sacrificing detection performance.