[Paper Review] Surveying You Only Look Once (YOLO) Multispectral Object Detection Advancements, Applications And Challenges

James Gallagher, Edward J. Oughton|arXiv (Cornell University)|Sep 3, 2024

Infrared Target Detection Methodologies4 citations

TL;DR

This paper presents a comprehensive meta-review of 200 multispectral YOLO papers, analyzing advancements in YOLO-based object detection across RGB and long-wave infrared (LWIR) fusion, with YOLOv5 being the most widely adapted variant. It identifies ground-based data collection as dominant (63%), highlights China as the leading research hub (58% of studies), and outlines key challenges and future directions in adaptive architectures, synthetic data generation, transfer learning, and multi-sensor fusion beyond RGB-LWIR.

ABSTRACT

Multispectral imaging and deep learning have emerged as powerful tools supporting diverse use cases from autonomous vehicles, to agriculture, infrastructure monitoring and environmental assessment. The combination of these technologies has led to significant advancements in object detection, classification, and segmentation tasks in the non-visible light spectrum. This paper considers 400 total papers, reviewing 200 in detail to provide an authoritative meta-review of multispectral imaging technologies, deep learning models, and their applications, considering the evolution and adaptation of You Only Look Once (YOLO) methods. Ground-based collection is the most prevalent approach, totaling 63% of the papers reviewed, although uncrewed aerial systems (UAS) for YOLO-multispectral applications have doubled since 2020. The most prevalent sensor fusion is Red-Green-Blue (RGB) with Long-Wave Infrared (LWIR), comprising 39% of the literature. YOLOv5 remains the most used variant for adaption to multispectral applications, consisting of 33% of all modified YOLO models reviewed. 58% of multispectral-YOLO research is being conducted in China, with broadly similar research quality to other countries (with a mean journal impact factor of 4.45 versus 4.36 for papers not originating from Chinese institutions). Future research needs to focus on (i) developing adaptive YOLO architectures capable of handling diverse spectral inputs that do not require extensive architectural modifications, (ii) exploring methods to generate large synthetic multispectral datasets, (iii) advancing multispectral YOLO transfer learning techniques to address dataset scarcity, and (iv) innovating fusion research with other sensor types beyond RGB and LWIR.

Motivation & Objective

• To provide a systematic meta-review of 200 out of 400 surveyed papers on YOLO-based multispectral object detection.
• To analyze the evolution, applications, and technical challenges of YOLO in multispectral imaging across diverse domains.
• To identify dominant data collection methods, sensor fusion patterns, and YOLO model variants in current research.
• To assess regional research distribution and quality, particularly comparing Chinese and non-Chinese research outputs.
• To outline future research directions, including adaptive architectures, synthetic data, transfer learning, and multi-sensor fusion.

Proposed method

• A systematic literature review was conducted on 400 papers, with 200 analyzed in detail to ensure authoritative coverage of multispectral YOLO research.
• The study categorized research by data collection method (e.g., ground-based, UAS), sensor fusion types (e.g., RGB-LWIR), and YOLO model variants (e.g., YOLOv5).
• Research output was analyzed by geographic origin, with journal impact factor used as a proxy for research quality.
• Trends in sensor fusion, model adaptation, and application domains were quantified and visualized across the reviewed literature.
• Future research needs were derived from gaps in architectural adaptability, data scarcity, and limited multi-sensor integration beyond RGB and LWIR.
• The analysis used statistical summaries of publication trends, model prevalence, and regional contributions to inform recommendations.

Experimental results

Research questions

• RQ1 What are the dominant data collection methods and sensor fusion configurations in YOLO-based multispectral object detection?
• RQ2 Which YOLO variant is most frequently adapted for multispectral applications, and how has this evolved over time?
• RQ3 What is the geographic distribution of multispectral YOLO research, and how does research quality compare across regions?
• RQ4 What are the key technical challenges and limitations in current multispectral YOLO research?
• RQ5 What future research directions are most critical to advancing the field of multispectral YOLO object detection?

Key findings

• Ground-based data collection is the most prevalent method, accounting for 63% of reviewed studies, with uncrewed aerial systems (UAS) usage doubling since 2020.
• RGB-LWIR fusion is the most common sensor combination, representing 39% of the literature.
• YOLOv5 is the most widely used YOLO variant for multispectral adaptation, accounting for 33% of all modified YOLO models reviewed.
• China leads in multispectral YOLO research, contributing 58% of all studies, with a mean journal impact factor of 4.45, comparable to non-Chinese research (4.36).
• A significant research gap exists in developing YOLO architectures that can handle diverse spectral inputs without extensive architectural rework.
• Future research must prioritize synthetic data generation, improved transfer learning for data-scarce scenarios, and fusion with sensors beyond RGB and LWIR.