[Paper Review] Applications of Deep Learning in Fish Habitat Monitoring: A Tutorial and Survey

Alzayat Saleh, Marcus Sheaves+2 more|arXiv (Cornell University)|Jun 11, 2022

Water Quality Monitoring Technologies4 citations

TL;DR

This paper presents a comprehensive tutorial and survey on applying deep learning (DL) to underwater fish habitat monitoring, covering key DL concepts, step-by-step model development, and state-of-the-art techniques for classification, counting, localization, and segmentation. It highlights the potential of DL to automate labor-intensive visual data analysis from underwater cameras, while addressing challenges like data scarcity, image quality, and model optimization in marine environments.

ABSTRACT

Marine ecosystems and their fish habitats are becoming increasingly important due to their integral role in providing a valuable food source and conservation outcomes. Due to their remote and difficult to access nature, marine environments and fish habitats are often monitored using underwater cameras. These cameras generate a massive volume of digital data, which cannot be efficiently analysed by current manual processing methods, which involve a human observer. DL is a cutting-edge AI technology that has demonstrated unprecedented performance in analysing visual data. Despite its application to a myriad of domains, its use in underwater fish habitat monitoring remains under explored. In this paper, we provide a tutorial that covers the key concepts of DL, which help the reader grasp a high-level understanding of how DL works. The tutorial also explains a step-by-step procedure on how DL algorithms should be developed for challenging applications such as underwater fish monitoring. In addition, we provide a comprehensive survey of key deep learning techniques for fish habitat monitoring including classification, counting, localization, and segmentation. Furthermore, we survey publicly available underwater fish datasets, and compare various DL techniques in the underwater fish monitoring domains. We also discuss some challenges and opportunities in the emerging field of deep learning for fish habitat processing. This paper is written to serve as a tutorial for marine scientists who would like to grasp a high-level understanding of DL, develop it for their applications by following our step-by-step tutorial, and see how it is evolving to facilitate their research efforts. At the same time, it is suitable for computer scientists who would like to survey state-of-the-art DL-based methodologies for fish habitat monitoring.

Motivation & Objective

• To provide marine scientists and computer scientists with a practical, step-by-step tutorial on applying deep learning to underwater fish habitat monitoring.
• To survey and compare state-of-the-art deep learning techniques—including classification, counting, localization, and semantic segmentation—for analyzing underwater fish imagery.
• To compile and review publicly available underwater fish datasets to support reproducible research and model benchmarking.
• To identify key challenges in deploying DL for underwater monitoring, such as poor visibility, image distortion, and limited annotated data.
• To outline future research directions, including self-supervised learning, automatic fish phenotyping, and real-time behavior tracking using low-power DL systems.

Proposed method

• Introduces foundational deep learning concepts, focusing on convolutional neural networks (CNNs) and their architecture for visual feature learning.
• Outlines a systematic, step-by-step pipeline for developing custom deep learning models tailored to underwater fish monitoring tasks.
• Reviews key deep learning techniques: instance segmentation (e.g., Mask R-CNN), object detection (e.g., YOLO, Faster R-CNN), and image classification (e.g., ResNets).
• Evaluates model training strategies such as data augmentation, transfer learning, dropout, and regularization to improve robustness on small or noisy datasets.
• Proposes integration of image enhancement techniques (e.g., color correction, noise reduction) to improve input quality before DL inference.
• Discusses the use of weakly supervised and self-supervised learning to reduce reliance on expensive human-annotated data.

Experimental results

Research questions

• RQ1 How can deep learning be systematically applied to automate the analysis of underwater fish habitat imagery collected via permanent or temporary camera systems?
• RQ2 What are the most effective deep learning architectures and training strategies for tasks such as fish detection, counting, and segmentation in challenging underwater conditions?
• RQ3 How do data quality issues—such as low visibility, color distortion, and noise—affect the performance of deep learning models in fish monitoring?
• RQ4 What publicly available datasets are currently available for training and benchmarking deep learning models in underwater fish monitoring?
• RQ5 What are the key challenges and future research opportunities in deploying real-time, low-power, and accurate deep learning systems for fish behavior tracking and phenotyping?

Key findings

• Deep learning significantly outperforms manual analysis in processing large volumes of underwater visual data, enabling scalable and efficient fish habitat monitoring.
• Transfer learning and data augmentation substantially improve model performance on limited or domain-specific underwater fish datasets.
• Image enhancement techniques, when combined with deep learning, can improve detection and segmentation accuracy by mitigating underwater visual degradation.
• Current public datasets for underwater fish monitoring are limited in size and annotation quality, creating a bottleneck for model generalization.
• Self-supervised and weakly supervised learning methods show promise in reducing dependency on costly human-annotated data in underwater applications.
• Real-time, low-power fish tracking systems remain a critical challenge, requiring further innovation in model efficiency and hardware integration.