[Paper Review] Promptable Foundation Models for SAR Remote Sensing: Adapting the Segment Anything Model for Snow Avalanche Segmentation

Riccardo Gelato, Carlo Sgaravatti+3 more|arXiv (Cornell University)|Jan 3, 2026

Cryospheric studies and observations0 citations

TL;DR

The paper adapts the Segment Anything Model (SAM) to SAR-based avalanche segmentation by using adapters, multi-encoder input handling, robust BB prompts, and a compute-efficient training scheme, boosting annotation speed and segmentation robustness.

ABSTRACT

Remote sensing solutions for avalanche segmentation and mapping are key to supporting risk forecasting and mitigation in mountain regions. Synthetic Aperture Radar (SAR) imagery from Sentinel-1 can be effectively used for this task, but training an effective detection model requires gathering a large dataset with high-quality annotations from domain experts, which is prohibitively time-consuming. In this work, we aim to facilitate and accelerate the annotation of SAR images for avalanche mapping. We build on the Segment Anything Model (SAM), a segmentation foundation model trained on natural images, and tailor it to Sentinel-1 SAR data. Adapting SAM to our use-case requires addressing several domain-specific challenges: (i) domain mismatch, since SAM was not trained on satellite/SAR imagery; (ii) input adaptation, because SAR products typically provide more than three channels, while SAM is constrained to RGB images; (iii) robustness to imprecise prompts that can affect target identification and degrade the segmentation quality, an issue exacerbated in small, low-contrast avalanches; and (iv) training efficiency, since standard fine-tuning is computationally demanding for SAM. We tackle these challenges through a combination of adapters to mitigate the domain gap, multiple encoders to handle multi-channel SAR inputs, prompt-engineering strategies to improve avalanche localization accuracy, and a training algorithm that limits the training time of the encoder, which is recognized as the major bottleneck. We integrate the resulting model into an annotation tool and show experimentally that it speeds up the annotation of SAR images.

Motivation & Objective

• Address domain shift between natural images and SAR data for avalanche segmentation.
• Develop a lightweight domain adaptation workflow that retrains only a small subset of SAM parameters.
• Enable multi-channel SAR input handling without changing the original SAM decoder architecture.
• Improve robustness to imprecise prompts and small avalanche targets through prompt engineering and training strategies.
• Demonstrate practical gains by integrating the model into a semi-automatic SAR annotation tool to speed up labeling.

Proposed method

• Fine-tune SAM using Adapters embedded in every ViT block to adapt the image encoder with ~7M trainable parameters (≈10% of the encoder) while keeping the decoder fine-tuned for binary avalanche segmentation.
• Train a decoder on the avalanche masks to preserve prompt-based segmentation capabilities.
• Implement a multi-encoder setup to process six SAR-related channels by using a primary encoder ([VV0, VV1, DEM]) and a secondary encoder ([VH0, VH1, SA]) with supervised alignment to the primary decoder’s space.
• Apply a BB (bounding box) prompt strategy augmented to be robust to prompt inaccuracies, including minimum enclosing boxes, random BB perturbations, and merging overlapping boxes.
• Introduce a resource-optimized training pipeline that reuses image embeddings so the decoder processes multiple prompts per image in parallel, reducing training time significantly.
• Employ a three-phase training procedure: Phase 1 domain adaptation with primary modality, Phase 2 alignment of secondary modality with frozen decoder, Phase 3 fusion of embeddings via a Selective Fusion Gate (SFG).
• Utilize a Dice loss to address class imbalance and a fixed probability threshold for final binarization, with data augmentations to combat speckle noise and overfitting.

Experimental results

Research questions

• RQ1 Can SAM be effectively adapted to SAR avalanche segmentation using a small, domain-specific adapter-based fine-tuning strategy?
• RQ2 Does a multi-encoder input architecture improve segmentation by leveraging complementary SAR channels and topographic data?
• RQ3 Does a BB-based prompting strategy with robustness augmentations maintain or improve segmentation performance under prompt imprecision and for small avalanches?
• RQ4 Can resource-efficient training that reuses image embeddings achieve practical speedups without sacrificing convergence or accuracy?
• RQ5 Does integration of the adapted SAM into a semi-automatic annotation tool accelerate SAR avalanche annotation workflows?

Key findings

• The adapters-based encoder tuning combined with decoder fine-tuning achieves competitive IoU for the avalanche class with significantly fewer trainable parameters (~7M) than full fine-tuning.
• A two-encoder (multi-channel) design with supervised embedding alignment and a Selective Fusion Gate improves segmentation by leveraging complementary SAR channels.
• A prompt robustness strategy using augmented BB prompts reduces sensitivity to prompt precision and enables prompt-free segmentation with full-image prompts.
• Resource optimization via shared image embeddings reduces training time by about 63% while maintaining convergence behavior.
• The integrated tool supports semi-automatic annotation, significantly speeding up the creation of avalanche masks in SAR imagery.