[Paper Review] FLAIR #1: semantic segmentation and domain adaptation dataset

Anatol Garioud, Stéphane Peillet+3 more|arXiv (Cornell University)|Jan 1, 2022

Geographic Information Systems Studies13 citations

TL;DR

This paper introduces FLAIR #1, a large-scale semantic segmentation and domain adaptation dataset comprising 20.3 billion annotated pixels across 50 spatial and temporal domains in metropolitan France, with 13 semantic classes at 0.20 m resolution. It presents a ResNet34/U-Net baseline model achieving a mean Intersection-over-Union (mIoU) of 0.5443±0.0014 on test patches, with limited improvement from metadata integration or data augmentation, highlighting challenges in generalization across heterogeneous land cover and imaging conditions.

ABSTRACT

The French National Institute of Geographical and Forest Information (IGN) has the mission to document and measure land-cover on French territory and provides referential geographical datasets, including high-resolution aerial images and topographic maps. The monitoring of land-cover plays a crucial role in land management and planning initiatives, which can have significant socio-economic and environmental impact. Together with remote sensing technologies, artificial intelligence (IA) promises to become a powerful tool in determining land-cover and its evolution. IGN is currently exploring the potential of IA in the production of high-resolution land cover maps. Notably, deep learning methods are employed to obtain a semantic segmentation of aerial images. However, territories as large as France imply heterogeneous contexts: variations in landscapes and image acquisition make it challenging to provide uniform, reliable and accurate results across all of France. The FLAIR-one dataset presented is part of the dataset currently used at IGN to establish the French national reference land cover map "Occupation du sol à grande échelle" (OCS- GE).

Motivation & Objective

• To address the challenge of domain shift in large-scale semantic segmentation of high-resolution aerial imagery across diverse French landscapes and imaging conditions.
• To provide a benchmark dataset for training and evaluating deep learning models in land cover mapping under real-world variability in terrain, seasonality, and sensor characteristics.
• To enable research into domain adaptation techniques by structuring the dataset into 50 spatial domains and multiple temporal domains reflecting real aerial survey constraints.
• To support the development of robust semantic segmentation models for national-scale land cover mapping using metadata and data augmentation strategies.

Proposed method

• The dataset comprises 77,412 patches (512×512) from 50 spatial domains (départements), each with multiple temporal acquisitions between April and November.
• Semantic segmentation is performed using a ResNet34/U-Net architecture trained on 61,712 patches and tested on 15,700 patches, with mIoU as the primary evaluation metric.
• Metadata integration is explored via a shallow MLP to encode spatial, temporal, and camera type information, with strategies like 'bottom' integration adding encoded metadata to the last encoder layer.
• Image data augmentation includes geometric transformations (flips, rotations) applied with 0.5 probability to improve generalization.
• The mIoU is computed as the average of per-class IoU scores, excluding the 'other' class which is treated as void.
• A confusion matrix is aggregated across all test patches before IoU calculation to ensure robust evaluation.

Experimental results

Research questions

• RQ1 How does model performance vary across diverse land cover classes in a large-scale, heterogeneous aerial imagery dataset with significant domain shift?
• RQ2 To what extent can metadata encoding and data augmentation improve semantic segmentation generalization in the presence of spatial, temporal, and radiometric variability?
• RQ3 What are the main sources of domain shift in high-resolution land cover segmentation, and how do they affect model generalization?
• RQ4 How do inter-class confusions manifest across semantically similar classes such as herbaceous vegetation and agricultural land?
• RQ5 Can a single deep learning model achieve robust performance across all 50 spatial and temporal domains in metropolitan France?

Key findings

• The baseline ResNet34/U-Net model achieves a mean Intersection-over-Union (mIoU) of 0.5443±0.0014 on the test set, indicating moderate overall performance across diverse land cover types.
• The highest-performing classes—buildings, water, and vineyards—achieve IoU scores around 0.8, indicating strong model confidence and distinct visual features.
• Classes such as bare soil, coniferous, brushwood, and plowed land show poor performance with IoU < 0.4, likely due to low frequency and visual similarity to other classes.
• Metadata integration via the 'bottom' strategy slightly improves mIoU to 0.5536±0.0014, but the improvement is marginal and not statistically significant across runs.
• Data augmentation with geometric transformations reduces mIoU to 0.5335±0.0038, suggesting potential negative impact or suboptimal augmentation strategy.
• Confusion matrices reveal significant inter-class confusion between semantically similar classes, such as herbaceous vegetation and agricultural land, and between bare soil and pervious surfaces.