[Paper Review] Backbones-Review: Feature Extraction Networks for Deep Learning and Deep Reinforcement Learning Approaches

Omar Elharrouss, Younes Akbari+2 more|arXiv (Cornell University)|Jun 16, 2022

Anomaly Detection Techniques and ApplicationsComputer Science144 references49 citations

TL;DR

A comprehensive survey of backbone networks (e.g., AlexNet, VGG, ResNet, DenseNet, EfficientNet, HRNet) used for feature extraction in DL and DRL, detailing architectures, tasks, and comparative insights.

ABSTRACT

To understand the real world using various types of data, Artificial Intelligence (AI) is the most used technique nowadays. While finding the pattern within the analyzed data represents the main task. This is performed by extracting representative features step, which is proceeded using the statistical algorithms or using some specific filters. However, the selection of useful features from large-scale data represented a crucial challenge. Now, with the development of convolution neural networks (CNNs), the feature extraction operation has become more automatic and easier. CNNs allow to work on large-scale size of data, as well as cover different scenarios for a specific task. For computer vision tasks, convolutional networks are used to extract features also for the other parts of a deep learning model. The selection of a suitable network for feature extraction or the other parts of a DL model is not random work. So, the implementation of such a model can be related to the target task as well as the computational complexity of it. Many networks have been proposed and become the famous networks used for any DL models in any AI task. These networks are exploited for feature extraction or at the beginning of any DL model which is named backbones. A backbone is a known network trained in many other tasks before and demonstrates its effectiveness. In this paper, an overview of the existing backbones, e.g. VGGs, ResNets, DenseNet, etc, is given with a detailed description. Also, a couple of computer vision tasks are discussed by providing a review of each task regarding the backbones used. In addition, a comparison in terms of performance is also provided, based on the backbone used for each task.

Motivation & Objective

• Survey and categorize backbone families used for feature extraction in DL and DRL.
• Discuss how backbones are selected for different computer vision tasks (classification, detection, segmentation, etc.).
• Provide comparative discussion on architectures, parameters, and computational considerations.
• Highlight challenges and future directions in backbone design and usage.

Proposed method

• Describe and categorize major backbone architectures (e.g., AlexNet, VGGs, ResNet, Inception, DenseNet, MobileNet, EfficientNet, HRNet, etc.).
• Summarize backbone characteristics: parameters, training task, and key architectural features.
• Review backbone deployment across computer vision tasks and DRL contexts.
• Provide qualitative comparisons and trends across tasks and backbones.

Experimental results

Research questions

• RQ1 What backbone architectures are most commonly used for feature extraction across DL and DRL tasks?
• RQ2 How do backbone choices impact performance and computational cost across image classification, object detection, crowd counting, and video summarization?
• RQ3 What are the observed trends and gaps in backbone design and usage, and what future directions are suggested?
• RQ4 How do backbones perform in DRL contexts compared to traditional DL tasks?

Key findings

• The paper enumerates and describes a wide range of backbones, including AlexNet, VGG-16/19, GoogleNet/Inception variants, ResNet families, DenseNet, Darknet, ShuffleNet, DetNet, SqueezeNet, MobileNet, WideResNet, EfficientNet, SWideRNet, Xception, and HRNet.
• Backbones are linked to specific CV tasks (image classification, object detection, crowd counting, video summarization, etc.) with attention to their strengths and contexts of use.
• Performance and complexity considerations (e.g., parameter counts, FLOPs, and suitability for devices with limited power) are discussed, highlighting trade-offs between accuracy and efficiency.
• The review notes evolving design trends such as depth vs. width, residual connections, multi-scale and high-resolution maintaining architectures, and efficient convolutional strategies for mobile and edge applications.
• A table summarizes backbone characteristics (year, parameters, trained task) for quick reference across common networks.