[Paper Review] Underwater Object Detection in the Era of Artificial Intelligence: Current, Challenge, and Future

Long Chen, Yuzhi Huang+6 more|arXiv (Cornell University)|Oct 8, 2024

Water Quality Monitoring Technologies4 citations

TL;DR

This survey provides a comprehensive analysis of AI-based underwater object detection (UOD), categorizing deep learning and traditional machine learning methods by learning strategy, datasets, features, and learning stages. It identifies key challenges—image degradation, small objects, noisy labels, and class imbalance—and evaluates state-of-the-art detectors using standardized benchmarks and two novel analysis tools, Diagnosis and TIDE, offering actionable insights for future research.

ABSTRACT

Underwater object detection (UOD), aiming to identify and localise the objects in underwater images or videos, presents significant challenges due to the optical distortion, water turbidity, and changing illumination in underwater scenes. In recent years, artificial intelligence (AI) based methods, especially deep learning methods, have shown promising performance in UOD. To further facilitate future advancements, we comprehensively study AI-based UOD. In this survey, we first categorise existing algorithms into traditional machine learning-based methods and deep learning-based methods, and summarise them by considering learning strategy, experimental dataset, utilised features or frameworks, and learning stage. Next, we discuss the potential challenges and suggest possible solutions and new directions. We also perform both quantitative and qualitative evaluations of mainstream algorithms across multiple benchmark datasets by considering the diverse and biased experimental setups. Finally, we introduce two off-the-shelf detection analysis tools, Diagnosis and TIDE, which well-examine the effects of object characteristics and various types of errors on detectors. These tools help identify the strengths and weaknesses of detectors, providing insigts for further improvement. The source codes, trained models, utilised datasets, detection results, and detection analysis tools are public available at \url{https://github.com/LongChenCV/UODReview}, and will be regularly updated.

Motivation & Objective

• To systematically categorize and analyze existing AI-based underwater object detection methods, including traditional machine learning and deep learning approaches.
• To identify and discuss major challenges in UOD, such as image degradation, small object detection, noisy labels, and class imbalance.
• To evaluate mainstream detection frameworks across multiple benchmark datasets using both quantitative and qualitative analysis.
• To introduce and demonstrate the utility of two detection analysis tools—Diagnosis and TIDE—for diagnosing detector weaknesses based on object characteristics and error types.
• To provide a public, regularly updated repository with source code, models, datasets, and results to enable fair and reproducible benchmarking.

Proposed method

• Categorizing existing UOD methods based on learning strategy, applicable datasets, utilized features or frameworks, and learning stages.
• Evaluating state-of-the-art detectors on multiple benchmark datasets (e.g., RUOD, DUO) using standardized metrics and comparative analysis.
• Applying the Diagnosis tool to analyze how object size and aspect ratio affect detection performance.
• Using the TIDE framework to assess the impact of different error types—such as background, localization, and missed ground truth—on detector performance.
• Comparing solutions in generic object detection (GOD) with those in UOD to identify research gaps and suggest improvements.
• Recommending high-quality, large-scale benchmarks for fair and consistent evaluation of new UOD algorithms.

Figure 1: Challenges such as (a) degraded image quality and small objects, (b) noisy labels, and (c) class imbalance in underwater object detection datasets significantly impair the performance of deep detection models. For instance, (c) highlights that the deep detector RepPoints [ 3 ] faces severe

Experimental results

Research questions

• RQ1 How do different learning strategies, network architectures, and loss functions impact the performance of underwater object detectors?
• RQ2 What are the primary challenges in underwater object detection, and how do they differ from those in generic object detection?
• RQ3 To what extent do object size, aspect ratio, and image quality degradation affect detection accuracy across different models?
• RQ4 How do various types of errors—such as background, localization, and missed ground truth—differ across detectors and datasets?
• RQ5 Can existing analysis tools like Diagnosis and TIDE effectively identify model weaknesses and guide future improvements in underwater object detection?

Key findings

• Most detectors exhibit significantly lower performance on extra-small (XS) and extra-wide (XW) objects, with detection accuracy dropping substantially on objects below the 10th percentile in size or aspect ratio.
• Detectors such as FoveaBox, Cascade R-CNN, RepPoints, and ATSS show higher rates of background and localization errors on the RUOD dataset, primarily due to severe color distortion and low contrast.
• On the DUO dataset, background errors are more pronounced due to blurring, causing seacucumbers to resemble sand and scallops to look like stones, leading to misclassification.
• RetinaNet produces more classification errors, likely due to focal loss overemphasizing hard-to-classify examples, which may lead to misclassification of easier instances.
• GridRCNN and DetectoRS show higher missed ground truth rates, attributed to their reliance on grid-based localization and complex feature extraction, which can fail on small or blurred objects.
• The use of Diagnosis and TIDE tools reveals that detectors generally perform best on medium-sized, square-shaped objects (1:1 aspect ratio), while performance degrades significantly for extreme aspect ratios due to square kernel and anchor box limitations.

Figure 2: The development of (a) AI in underwater object detection and (b) generic artificial intelligence.