[Paper Review] Deep Fusion of Local and Non-Local Features for Precision Landslide Recognition

Qing Zhu, Chen Lin+4 more|arXiv (Cornell University)|Feb 20, 2020

Landslides and related hazards24 references4 citations

TL;DR

This paper proposes DA-U-Net, a U-Net-based model that fuses local and non-local features via dilated convolutions with atrous spatial pyramid pooling and a scale attention mechanism for improved landslide segmentation in post-hazard UAV imagery. The method achieves state-of-the-art performance, with a 59.41% IoU, significantly outperforming baseline models like U-Net and DeeplabV3+.

ABSTRACT

Precision mapping of landslide inventory is crucial for hazard mitigation. Most landslides generally co-exist with other confusing geological features, and the presence of such areas can only be inferred unambiguously at a large scale. In addition, local information is also important for the preservation of object boundaries. Aiming to solve this problem, this paper proposes an effective approach to fuse both local and non-local features to surmount the contextual problem. Built upon the U-Net architecture that is widely adopted in the remote sensing community, we utilize two additional modules. The first one uses dilated convolution and the corresponding atrous spatial pyramid pooling, which enlarged the receptive field without sacrificing spatial resolution or increasing memory usage. The second uses a scale attention mechanism to guide the up-sampling of features from the coarse level by a learned weight map. In implementation, the computational overhead against the original U-Net was only a few convolutional layers. Experimental evaluations revealed that the proposed method outperformed state-of-the-art general-purpose semantic segmentation approaches. Furthermore, ablation studies have shown that the two models afforded extensive enhancements in landslide-recognition performance.

Motivation & Objective

• To address the challenge of ambiguous landslide detection in complex terrain where landslides coexist with spectrally similar features like roads and bare earth.
• To improve semantic segmentation accuracy by fusing local receptive field information with non-local contextual understanding in remote sensing imagery.
• To preserve fine-grained boundaries during up-sampling by mitigating feature loss from downsampling in U-Net architectures.
• To develop a lightweight, efficient enhancement to U-Net that maintains low memory usage while boosting performance on landslide recognition tasks.

Proposed method

• Introduces a dilated convolution and atrous spatial pyramid pooling (ASPP) module in the U-Net bottleneck to expand the receptive field without reducing spatial resolution or increasing memory usage.
• Employs a scale attention mechanism that learns a weight map to guide feature up-sampling, suppressing irrelevant coarse-scale features and enhancing relevant high-resolution features.
• Maintains the U-Net’s encoder-decoder structure but enhances both encoder and decoder paths with two lightweight modules: ASPP for contextual feature aggregation and attention for feature refinement during up-sampling.
• Uses end-to-end training with binary cross-entropy loss to optimize segmentation of landslide vs. non-landslide pixels in post-hazard UAV images.
• Leverages multi-scale features through dilated convolutions with multiple rates to capture diverse spatial contexts without downsampling.
• Applies the attention module in the skip connections to dynamically reweight features based on their relevance, improving boundary preservation.

Experimental results

Research questions

• RQ1 Can the fusion of local and non-local features significantly improve landslide detection accuracy in ambiguous, high-spectral-similarity regions such as roads and bare earth?
• RQ2 To what extent does the attention-guided up-sampling mechanism reduce boundary loss and improve segmentation fidelity compared to standard U-Net up-sampling?
• RQ3 How do dilated convolutions with ASPP contribute to contextual understanding without sacrificing spatial resolution or increasing computational cost?
• RQ4 Does the combination of both modules (dilation + attention) yield synergistic improvements over individual components in landslide segmentation?
• RQ5 How does the proposed DA-U-Net compare to state-of-the-art models like DeeplabV3+ and PSPNet in terms of IoU, precision, recall, and F-score on landslide detection benchmarks?

Key findings

• The proposed DA-U-Net achieved a 59.41% Intersection over Union (IoU), significantly outperforming the baseline U-Net (48.18%) and DeeplabV3+ (57.25%).
• The ablation study confirmed that both the dilated convolution + ASPP module and the attention module independently improved performance, with the combined model yielding the highest IoU.
• On confusing regions such as roads and bare earth, DA-U-Net demonstrated superior generalization, reducing false positives compared to FCN, U-Net, and PSPNet.
• The attention-guided up-sampling module effectively suppressed noise and irrelevant features from coarse layers, resulting in sharper and more accurate landslide boundary predictions.
• The method achieved state-of-the-art performance with minimal computational overhead, adding only a few convolutional layers to the original U-Net architecture.
• The model showed robustness in detecting landslides in complex, ambiguous environments where spectral similarity with background features typically hinders segmentation.