[Paper Review] Semi-Supervised Semantic Segmentation in Earth Observation: The MiniFrance Suite, Dataset Analysis and Multi-task Network Study

Javiera Castillo-Navarro, Bertrand Le Saux+3 more|arXiv (Cornell University)|Oct 15, 2020

Remote-Sensing Image ClassificationEngineering61 references71 citations

TL;DR

This paper introduces the MiniFrance dataset for semi-supervised semantic segmentation in Earth Observation, analyzes dataset representativeness and appearance, and studies multi-task semi-supervised networks as baselines.

ABSTRACT

The development of semi-supervised learning techniques is essential to enhance the generalization capacities of machine learning algorithms. Indeed, raw image data are abundant while labels are scarce, therefore it is crucial to leverage unlabeled inputs to build better models. The availability of large databases have been key for the development of learning algorithms with high level performance. Despite the major role of machine learning in Earth Observation to derive products such as land cover maps, datasets in the field are still limited, either because of modest surface coverage, lack of variety of scenes or restricted classes to identify. We introduce a novel large-scale dataset for semi-supervised semantic segmentation in Earth Observation, the MiniFrance suite. MiniFrance has several unprecedented properties: it is large-scale, containing over 2000 very high resolution aerial images, accounting for more than 200 billions samples (pixels); it is varied, covering 16 conurbations in France, with various climates, different landscapes, and urban as well as countryside scenes; and it is challenging, considering land use classes with high-level semantics. Nevertheless, the most distinctive quality of MiniFrance is being the only dataset in the field especially designed for semi-supervised learning: it contains labeled and unlabeled images in its training partition, which reproduces a life-like scenario. Along with this dataset, we present tools for data representativeness analysis in terms of appearance similarity and a thorough study of MiniFrance data, demonstrating that it is suitable for learning and generalizes well in a semi-supervised setting. Finally, we present semi-supervised deep architectures based on multi-task learning and the first experiments on MiniFrance.

Motivation & Objective

• Introduce MiniFrance, a large-scale dataset for semi-supervised semantic segmentation in Earth Observation (EO).
• Analyze data representativeness and appearance similarity to define suitable training/test splits.
• Propose semi-supervised multi-task neural architectures and losses as baselines for EO segmentation.
• Evaluate how unlabeled data improves generalization and provide public baselines for future work.

Proposed method

• Describe MiniFrance properties: large-scale, 2,121 aerial images, 50 cm/px, 16 conurbations, 53,000 km2, 14 land-use classes.
• Develop data representativeness tools to assess appearance similarity and domain coverage across cities (CNN feature extraction, t-SNE, one-class SVM, IoU/IoT metrics).
• Propose semi-supervised network architectures that jointly optimize supervised segmentation and unsupervised tasks (multi-task learning) with shared parameters.
• Formulate semi-supervised loss as L(x) = Ls(phi_s(x), y) + lambda Lu(phi_u(x), x).
• Introduce BerundaNet (early/late) and W-Net architectures as multi-task baselines for semi-supervised learning in EO.

Experimental results

Research questions

• RQ1 How representative are the labeled/unlabeled splits across cities for learning robust EO segmentation models?
• RQ2 Can unlabeled data improve segmentation generalization in EO when combined with labeled data in a multi-task framework?
• RQ3 What are effective network architectures and loss formulations for semi-supervised semantic segmentation in aerial imagery?
• RQ4 How do appearance and class distribution across locations affect learning and generalization in MiniFrance?

Key findings

• MiniFrance is a large-scale, diverse EO dataset with 2,121 RGB tiles (10,000 x 10,000 px) at 50 cm/px and 53,000 km2, covering 16 conurbations with 14 land-use classes.
• The dataset includes labeled and unlabeled training partitions to mimic realistic semi-supervised learning scenarios and eight-city test split.
• Appearance and class representativeness analyses show that training cities cover test appearances to a meaningful extent, though no single city mirrors another exactly.
• Two analysis tools (IoU and IoT) quantify appearance similarity and coverage between cities, guiding partition design for semi-supervised learning.
• TinyMiniFrance is introduced as a computationally affordable subset (3,500 images) preserving class proportions for rapid prototyping.
• Proposed BerundaNet and W-Net multi-task architectures with unsupervised losses can serve as baselines for semi-supervised EO segmentation on MiniFrance.
• The study provides baselines and methodological tools to promote future semi-supervised work on Earth observation data.