[论文解读] Remote Sensing Images Semantic Segmentation with General Remote Sensing Vision Model via a Self-Supervised Contrastive Learning Method.

Haifeng Li, Yi Li+5 more|arXiv (Cornell University)|Jun 20, 2021

Remote-Sensing Image Classification参考文献 39被引用 5

一句话总结

该论文提出GLCNet，一种用于遥感语义分割的自监督对比学习框架，通过联合优化全局风格与局部特征匹配，提升像素级表征学习效果。仅使用1%的标注数据，该方法在ISPRS Potsdam数据集上将Kappa值提升6%，在DeepGlobe数据集上提升3%，达到当前最优性能。

ABSTRACT

A new learning paradigm, self-supervised learning (SSL), can be used to solve such problems by pre-training a general model with large unlabeled images and then fine-tuning on a downstream task with very few labeled samples. Contrastive learning is a typical method of SSL, which can learn general invariant features. However, most of the existing contrastive learning is designed for classification tasks to obtain an image-level representation, which may be sub-optimal for semantic segmentation tasks requiring pixel-level discrimination. Therefore, we propose Global style and Local matching Contrastive Learning Network (GLCNet) for remote sensing semantic segmentation. Specifically, the global style contrastive module is used to learn an image-level representation better, as we consider the style features can better represent the overall image features; The local features matching contrastive module is designed to learn representations of local regions which is beneficial for semantic segmentation. We evaluate four remote sensing semantic segmentation datasets, and the experimental results show that our method mostly outperforms state-of-the-art self-supervised methods and ImageNet pre-training. Specifically, with 1\% annotation from the original dataset, our approach improves Kappa by 6\% on the ISPRS Potsdam dataset and 3\% on Deep Globe Land Cover Classification dataset relative to the existing baseline. Moreover, our method outperforms supervised learning when there are some differences between the datasets of upstream tasks and downstream tasks. Our study promotes the development of self-supervised learning in the field of remote sensing semantic segmentation. The source code is available at this https URL.

研究动机与目标

• 解决遥感图像语义分割中标注数据有限的挑战。
• 提升自监督对比学习在遥感任务中像素级表征学习的效果。
• 开发一种统一框架，同时利用全局图像风格与局部区域特征，以提升分割性能。
• 降低对ImageNet预训练以及遥感应用中下游领域偏移的依赖。

提出的方法

• GLCNet引入全局风格对比模块，通过强调风格特征来学习图像级表征，从而提升整体场景理解能力。
• 采用局部特征匹配对比模块，在局部区域层面学习判别性表征，这对像素级分割至关重要。
• 模型使用对比学习方法，最大化同一图像不同增强视图（正样本对）的一致性，同时最小化不同图像视图（负样本对）的相似性。
• 在大规模未标注遥感图像上进行预训练，随后在下游分割任务上使用极少标注数据进行微调。
• 通过结合全局与局部对比目标的对比损失，端到端训练整个框架。

实验结果

研究问题

• RQ1 自监督对比学习能否有效提升遥感语义分割中的像素级表征学习？
• RQ2 结合全局风格与局部特征匹配是否优于现有为图像分类设计的对比学习方法？
• RQ3 在少样本学习设置下，GLCNet相较于ImageNet预训练和当前最优自监督方法表现如何？
• RQ4 当预训练与微调数据集之间存在分布偏移时，GLCNet是否具备良好的跨域泛化能力？

主要发现

• 在ISPRS Potsdam数据集上，仅使用原始标注数据的1%时，GLCNet将Kappa值提升了6%。
• 在DeepGlobe土地覆盖分类数据集上，相同少样本设置下，该方法相比现有基线模型将Kappa值提升了3%。
• 在所有评估的遥感分割基准上，该模型均优于ImageNet预训练方法。
• 当上游与下游数据集之间存在领域偏移时，GLCNet的表现甚至超过监督学习，展现出强大的泛化能力。
• 消融实验证实，全局风格模块与局部匹配模块均对性能提升有显著贡献。