[论文解读] SpaceNet 6: Multi-Sensor All Weather Mapping Dataset

Jacob Shermeyer, David B. Hogan+9 more|arXiv (Cornell University)|Apr 14, 2020

Video Surveillance and Tracking Methods参考文献 34被引用 13

一句话总结

本文介绍了SpaceNet 6多传感器全天候制图（MSAW）数据集，该数据集包含120 km²的高分辨率（亚1米地面采样距离）SAR与光学影像，以及48,000个标注的建筑物轮廓。结果表明，从光学数据进行迁移学习可显著提升基于SAR的建筑物轮廓分割性能（F₁ = 0.21），相比仅在SAR上训练（F₁ = 0.135）提升55%，凸显了多模态数据在全天候地理空间分析中的价值。

ABSTRACT

Within the remote sensing domain, a diverse set of acquisition modalities exist, each with their own unique strengths and weaknesses. Yet, most of the current literature and open datasets only deal with electro-optical (optical) data for different detection and segmentation tasks at high spatial resolutions. optical data is often the preferred choice for geospatial applications, but requires clear skies and little cloud cover to work well. Conversely, Synthetic Aperture Radar (SAR) sensors have the unique capability to penetrate clouds and collect during all weather, day and night conditions. Consequently, SAR data are particularly valuable in the quest to aid disaster response, when weather and cloud cover can obstruct traditional optical sensors. Despite all of these advantages, there is little open data available to researchers to explore the effectiveness of SAR for such applications, particularly at very-high spatial resolutions, i.e. <1m Ground Sample Distance (GSD). To address this problem, we present an open Multi-Sensor All Weather Mapping (MSAW) dataset and challenge, which features two collection modalities (both SAR and optical). The dataset and challenge focus on mapping and building footprint extraction using a combination of these data sources. MSAW covers 120 km^2 over multiple overlapping collects and is annotated with over 48,000 unique building footprints labels, enabling the creation and evaluation of mapping algorithms for multi-modal data. We present a baseline and benchmark for building footprint extraction with SAR data and find that state-of-the-art segmentation models pre-trained on optical data, and then trained on SAR (F1 score of 0.21) outperform those trained on SAR data alone (F1 score of 0.135).

研究动机与目标

• 为解决多模态地理空间分析中缺乏公开、高分辨率SAR与光学数据集的问题。
• 支持利用合成孔径雷达（SAR）和光学数据进行全天候建筑物轮廓提取的研究。
• 为在具有挑战性的现实空基成像条件下评估多模态分割模型提供基准。
• 通过支持在持续云层覆盖下实现稳健制图，推动灾害响应研究。

提出的方法

• MSAW数据集包含在多个地点获取的120 km²重叠SAR与光学卫星影像。
• 建筑物轮廓通过高精度LiDAR衍生的高度估计进行标注，确保几何一致性与对象级标注。
• 采用迁移学习训练基线语义分割模型：先在光学数据上预训练，再在SAR数据上微调。
• SAR影像包含三种极化方式（HH、VV、HV），通过RGB合成图增强特征感知。
• 使用F₁分数、IoU和召回率评估性能，并按建筑物尺寸与高度分层分析。
• 使用随机森林模型评估建筑物属性（面积、高度、体积）对分割IoU分数的预测能力。

Figure 1: Building footprints outlined in red overlaid upon visualized SAR Intensity with three polarizations (HH, VV, HV) displayed through the Red, Green, and Blue color channels.

实验结果

研究问题

• RQ1 从光学数据进行迁移学习是否能显著提升高分辨率SAR影像中建筑物轮廓分割的性能？
• RQ2 建筑物尺寸与高度如何影响其在SAR影像中的可检测性？
• RQ3 几何失真（如斜距展叠与前向压缩）在多大程度上会降低高大建筑物的分割性能？
• RQ4 建筑物属性（如面积、高度、体积）在多大程度上可预测SAR数据中分割模型的性能（IoU）？
• RQ5 单模态（仅SAR）与多模态（SAR + 光学）训练在建筑物轮廓提取中的性能差距如何？

主要发现

• 从光学预训练迁移至SAR数据的迁移学习方法实现了F₁分数0.21，相比仅在SAR上训练（F₁ = 0.135）提升了55%。
• 当建筑物面积约为100 m²时，建筑物轮廓检测的召回率降至0.1，表明小尺寸建筑物在SAR影像中几乎无法检测。
• 对于较大建筑物，性能在召回率0.5处趋于平稳，但超过20米高的建筑物因斜距展叠等几何失真导致性能下降。
• 尽管面积较大（平均1400 m²），高度超过30米的建筑物平均召回率仍低于0.2，凸显雷达特异性失真影响显著。
• 建筑物高度是分割IoU的最强预测特征（重要性 = 0.39），即使未在基线模型中显式使用，因其与面积和失真效应强相关。
• 随机森林模型在从建筑物属性预测IoU时达到R² = 0.91，证实面积与高度是分割难度的强指标。

Figure 2: Examples of layover in urban SAR imagery. In this detail of Rotterdam, there is land to the north (top) and water to the south (bottom). This image was recorded by aircraft from the south. The three skyscrapers near the riverbank appear to jut into the water because of layover.