[Paper Review] NL-LinkNet: Toward Lighter but More Accurate Road Extraction with Non-Local Operations

Yooseung Wang, Junghoon Seo+1 more|arXiv (Cornell University)|Aug 22, 2019

Automated Road and Building Extraction26 references9 citations

TL;DR

This paper proposes NL-LinkNet, a lightweight yet highly accurate U-Net-based architecture for road extraction from very high-resolution satellite images, leveraging non-local operations to capture long-range contextual dependencies. By integrating non-local blocks into the encoder-decoder structure, NL-LinkNet achieves 65.00% mIOU—surpassing the official DeepGlobe Challenge winner (D-LinkNet) with 43% fewer parameters, lower GFLOPs, and faster convergence.

ABSTRACT

Road extraction from very high resolution satellite (VHR) images is one of the most important topics in the field of remote sensing. In this paper, we propose an efficient Non-Local LinkNet with non-local blocks that can grasp relations between global features. This enables each spatial feature point to refer to all other contextual information and results in more accurate road segmentation. In detail, our single model without any post-processing like CRF refinement, performed better than any other published state-of-the-art ensemble model in the official DeepGlobe Challenge. Moreover, our NL-LinkNet beat the D-LinkNet, the winner of the DeepGlobe challenge, with 43 \% less parameters, less giga floating-point operations per seconds (GFLOPs) and shorter training convergence time. We also present empirical analyses on the proper usages of non-local blocks for the baseline model.

Motivation & Objective

• To address the challenge of limited long-range context in CNN-based road segmentation due to small receptive fields.
• To improve road extraction accuracy in VHR satellite images where roads are partially occluded by shadows, trees, or buildings.
• To develop a lighter model than existing SOTA methods without relying on post-processing like CRF.
• To empirically evaluate the impact of non-local block placement and pairwise function design in road segmentation.

Proposed method

• Introduces a non-local block that computes feature representations as a weighted sum over all spatial positions, enabling global context modeling.
• Integrates non-local blocks into the LinkNet architecture at different stages (e.g., stage 3 or 4) to enhance feature representation without increasing model depth.
• Employs the embedded Gaussian pairwise function for computing attention weights between features, improving modeling of long-range dependencies.
• Uses a residual encoder (ResNet-34) as the backbone, with skip connections preserving spatial resolution for precise segmentation.
• Applies end-to-end training with cross-entropy and dice loss to optimize segmentation performance.
• Evaluates multiple configurations: single vs. double non-local blocks, and different pairwise functions (dot-product, Gaussian, embedded Gaussian).

Experimental results

Research questions

• RQ1 Can non-local operations significantly improve road segmentation accuracy in VHR satellite images compared to standard CNNs with limited receptive fields?
• RQ2 Where in the network architecture is the placement of non-local blocks most effective for road extraction?
• RQ3 Which pairwise function (e.g., dot-product, Gaussian, embedded Gaussian) yields the best performance in non-local attention for road segmentation?
• RQ4 Can a single non-local model outperform ensemble-based SOTA models without post-processing like CRF?
• RQ5 Does the integration of non-local blocks lead to a more parameter-efficient and faster-converging model compared to dilated convolution-based models?

Key findings

• NL-LinkNet achieved 65.00% mIOU on the DeepGlobe Challenge test set, outperforming the official 1st-place solution (D-LinkNet) with 43% fewer parameters.
• The model achieved lower GFLOPs (124.88 B) and faster training convergence than D-LinkNet (134.45 B GFLOPs) and PSANet (131.54 B GFLOPs).
• Adding non-local blocks at stage 4 (NL4-LinkNet) improved mIOU by 1.33% over the baseline, while double non-local blocks (NL34-LinkNet) achieved 1.52% improvement.
• The embedded Gaussian pairwise function yielded the highest mIOU (64.59%) among all configurations, outperforming dot-product (64.04%) and Gaussian (64.23%) functions.
• NL-LinkNet surpassed all previously published ensemble models in the DeepGlobe Challenge, demonstrating that a single model can exceed ensemble performance without CRF refinement.
• Visual comparisons confirmed that NL-LinkNet better detects roads in complex urban areas, winding roads, and roundabouts where local methods fail due to occlusions.