[Paper Review] OilSAM2: Memory-Augmented SAM2 for Scalable SAR Oil Spill Detection

Shuaiyu Chen, Ming Yin+3 more|arXiv (Cornell University)|Mar 10, 2026

Oil Spill Detection and Mitigation0 citations

TL;DR

OilSAM2 proposes a memory-augmented SAM2 framework with a hierarchical, multi-scale memory bank and a structure–semantic consistent memory update to enable cross-image, prompt-driven segmentation of oil spills in unordered SAR image collections.

ABSTRACT

Segmenting oil spills from Synthetic Aperture Radar (SAR) imagery remains challenging due to severe appearance variability, scale heterogeneity, and the absence of temporal continuity in real world monitoring scenarios. While foundation models such as Segment Anything (SAM) enable prompt driven segmentation, existing SAM based approaches operate on single images and cannot effectively reuse information across scenes. Memory augmented variants (e.g., SAM2) further assume temporal coherence, making them prone to semantic drift when applied to unordered SAR image collections. We propose OilSAM2, a memory augmented segmentation framework tailored for unordered SAR oil spill monitoring. OilSAM2 introduces a hierarchical feature aware multi scale memory bank that explicitly models texture, structure, and semantic level representations, enabling robust cross image information reuse. To mitigate memory drift, we further propose a structure semantic consistent memory update strategy that selectively refreshes memory based on semantic discrepancy and structural variation.Experiments on two public SAR oil spill datasets demonstrate that OilSAM2 achieves state of the art segmentation performance, delivering stable and accurate results under noisy SAR monitoring scenarios. The source code is available at https://github.com/Chenshuaiyu1120/OILSAM2.

Motivation & Objective

• Address the challenge of oil spill segmentation in unordered SAR imagery with variable appearances and scales.
• Leverage a memory-augmented SAM2 framework to reuse information across images while mitigating semantic drift.
• Introduce a hierarchical texture/structure/semantic memory bank for robust cross-image information fusion.
• Develop a structure–semantic consistent memory update strategy to refresh memory based on reliability cues.

Proposed method

• Build a hierarchical multi-level memory bank (texture, structure, semantic) integrated with a SAM2-based prompt-driven segmentation framework.
• Perform scale-adaptive memory fusion by retrieving and weighting level-specific memories before fusing into a common representation.
• Introduce a structure–semantic consistent memory update mechanism that triggers memory refresh based on semantic and structural discrepancies (cosine semantic distance and gradient-based structure change).
• Train scale adapters, memory components, and the mask decoder while freezing the original SAM2 modules.
• Optimize segmentation with a weighted binary cross-entropy loss on the two-class oil spill task.

Fig. 1 : Overview of the proposed OilSAM2 framework. Given an input SAR image and user prompt, hierarchical features are extracted and organized into texture-, structure-, and semantic-level representations. Each level interacts with a corresponding memory group via scale-wise attention. The retriev

Experimental results

Research questions

• RQ1 Can memory-augmented segmentation across unordered SAR images improve oil spill delineation compared to single-image SAM-based methods?
• RQ2 Does a multi-level memory bank capturing texture, structure, and semantic cues enhance robustness to sea state, speckle noise, and look-alike phenomena?
• RQ3 Does a structure–semantic consistent memory update reduce semantic drift when processing unordered SAR image collections?

Key findings

• OilSAM2 achieves state-of-the-art performance on two public SAR oil spill datasets, outperforming SAM-based and other baselines.
• On M4D, OilSAM2 yields a notable IoU improvement for oil spills and strong overall mIoU.
• On SOS, OilSAM2 attains the highest mIoU and robust F1-score/recall across PALSAR and Sentinel-1 modalities.
• A multi-scale memory bank plus structure–semantic update provides larger gains than individual components in ablations.
• The approach demonstrates improved boundary delineation and detection of small/fragments under diverse sea states.

Fig. 2 : Qualitative results of the ablation study(On SOS Dataset).