[Paper Review] xView3-SAR: Detecting Dark Fishing Activity Using Synthetic Aperture Radar Imagery

Fernando Paolo, Tsu-ting Tim Lin+6 more|arXiv (Cornell University)|Jun 2, 2022

Underwater Acoustics Research17 citations

TL;DR

This paper introduces xView3-SAR, the largest publicly available dataset for detecting and characterizing dark vessels in Synthetic Aperture Radar (SAR) imagery, combining AIS data with expert-verified annotations across 991 high-resolution SAR images. The dataset enables large-scale machine learning for anti-illegal, unreported, and unregulated (IUU) fishing, with a companion challenge demonstrating state-of-the-art performance in ship detection, classification, and length estimation using SAR-specific data augmentation and deep learning models.

ABSTRACT

Unsustainable fishing practices worldwide pose a major threat to marine resources and ecosystems. Identifying vessels that do not show up in conventional monitoring systems -- known as ``dark vessels'' -- is key to managing and securing the health of marine environments. With the rise of satellite-based synthetic aperture radar (SAR) imaging and modern machine learning (ML), it is now possible to automate detection of dark vessels day or night, under all-weather conditions. SAR images, however, require a domain-specific treatment and are not widely accessible to the ML community. Maritime objects (vessels and offshore infrastructure) are relatively small and sparse, challenging traditional computer vision approaches. We present the largest labeled dataset for training ML models to detect and characterize vessels and ocean structures in SAR imagery. xView3-SAR consists of nearly 1,000 analysis-ready SAR images from the Sentinel-1 mission that are, on average, 29,400-by-24,400 pixels each. The images are annotated using a combination of automated and manual analysis. Co-located bathymetry and wind state rasters accompany every SAR image. We also provide an overview of the xView3 Computer Vision Challenge, an international competition using xView3-SAR for ship detection and characterization at large scale. We release the data (\href{https://iuu.xview.us/}{https://iuu.xview.us/}) and code (\href{https://github.com/DIUx-xView}{https://github.com/DIUx-xView}) to support ongoing development and evaluation of ML approaches for this important application.

Motivation & Objective

• To address the critical challenge of detecting dark vessels—fishing vessels that do not broadcast AIS signals—using satellite-based Synthetic Aperture Radar (SAR) imagery.
• To overcome the limitations of existing datasets by creating the largest and most comprehensive SAR-based maritime object detection dataset with rich, multi-task annotations.
• To enable large-scale machine learning for anti-IUU fishing by releasing analysis-ready SAR images, detailed annotations, and ancillary environmental rasters (bathymetry, wind conditions).
• To promote cross-disciplinary research by organizing the xView3 Computer Vision Challenge, fostering innovation in SAR-specific computer vision techniques.
• To support practical deployment of AI tools in government and non-governmental organizations by releasing open-source code, models, and tooling.

Proposed method

• The dataset was constructed using a hybrid labeling approach combining automated AIS-to-SAR matching with manual verification by expert analysts to ensure accuracy of vessel detections.
• Each of the 991 SAR images, sourced from the Sentinel-1 mission, is approximately 29,400 × 24,400 pixels and includes dual-polarization (VV and VH) data.
• Annotations include bounding boxes, vessel type classification, and length estimation (in pixels), with a focus on detecting small, sparse objects in cluttered, noisy SAR backgrounds.
• Co-located bathymetry and wind condition rasters were included to provide environmental context for improved model generalization and interpretability.
• A custom data augmentation pipeline was developed and extended from Albumentations, incorporating SAR-specific transformations such as noise injection and apodization in Fourier space to mimic speckle patterns.
• The xView3 Computer Vision Challenge used the dataset to benchmark models on a multitask problem: ship detection, classification, and length regression, with top models leveraging single-stage architectures and imbalanced class training.

Experimental results

Research questions

• RQ1 How can large-scale, high-resolution SAR imagery be effectively used to detect and characterize dark fishing vessels that evade conventional AIS monitoring?
• RQ2 What are the most effective data augmentation and model architecture strategies for improving object detection performance in SAR imagery, given its unique noise and speckle characteristics?
• RQ3 To what extent can hybrid labeling—combining AIS data with expert human verification—produce reliable, scalable annotations for maritime objects in SAR?
• RQ4 How do environmental context rasters (bathymetry and wind conditions) improve the performance and robustness of SAR-based vessel detection models?
• RQ5 What are the key limitations of current deep learning models in detecting vessels near shorelines, where SAR artifacts are more prevalent?

Key findings

• xView3-SAR is the largest SAR-based maritime object detection dataset to date, covering 43.2 million square kilometers across 991 full-resolution Sentinel-1 SAR images, with 243,018 verified maritime objects.
• The dataset includes dual-band (VV and VH) SAR imagery with co-located bathymetry and wind condition rasters, providing critical environmental context for model training.
• The xView3 Computer Vision Challenge attracted global participation, with top models achieving significant improvements in detection, classification, and length estimation over baseline methods.
• All top-performing models in the challenge employed SAR-specific data augmentation techniques, such as noise injection and apodization in Fourier space, to mitigate speckle and improve robustness.
• Despite advances, all models struggled with vessel detection near shorelines due to artifacts from multi-path effects and complex scattering, highlighting a key open challenge.
• The released code, models, and tooling from the challenge have already been deployed by government and NGO partners to support real-world anti-IUU fishing operations.