[Paper Review] Learning to Measure Change: Fully Convolutional Siamese Metric Networks for Scene Change Detection

Enqiang Guo, Xinsha Fu+5 more|ArXiv.org|Oct 22, 2018

Remote-Sensing Image Classification49 references72 citations

TL;DR

This paper introduces CosimNet, a fully convolutional siamese metric network that directly measures pixelwise changes between image pairs using learned metric learning with contrastive and Thresholded Contrastive Loss to distinguish semantic changes from noisy changes under illumination, viewpoint, and zoom variations.

ABSTRACT

A critical challenge problem of scene change detection is that noisy changes generated by varying illumination, shadows and camera viewpoint make variances of a scene difficult to define and measure since the noisy changes and semantic ones are entangled. Following the intuitive idea of detecting changes by directly comparing dissimilarities between a pair of features, we propose a novel fully Convolutional siamese metric Network(CosimNet) to measure changes by customizing implicit metrics. To learn more discriminative metrics, we utilize contrastive loss to reduce the distance between the unchanged feature pairs and to enlarge the distance between the changed feature pairs. Specifically, to address the issue of large viewpoint differences, we propose Thresholded Contrastive Loss (TCL) with a more tolerant strategy to punish noisy changes. We demonstrate the effectiveness of the proposed approach with experiments on three challenging datasets: CDnet, PCD2015, and VL-CMU-CD. Our approach is robust to lots of challenging conditions, such as illumination changes, large viewpoint difference caused by camera motion and zooming. In addition, we incorporate the distance metric into the segmentation framework and validate the effectiveness through visualization of change maps and feature distribution. The source code is available at https://github.com/gmayday1997/ChangeDet.

Motivation & Objective

• Motivate changing detection as a dissimilarity measurement problem rather than classification-only approaches.
• Develop an end-to-end trainable framework that learns an implicit metric for change detection.
• Reduce sensitivity to noisy changes caused by illumination, shadows, and camera viewpoint differences.
• Incorporate a distance metric into FCN-based change detection and validate via multiple datasets.

Proposed method

• Use a fully convolutional siamese network to extract paired features from image pairs.
• Apply a predefined distance metric (e.g., L2 or cosine) on normalized features to produce a change map.
• Train with contrastive loss to pull together unchanged pairs and push apart changed pairs.
• Introduce Thresholded Contrastive Loss (TCL) to tolerate large viewpoint differences.
• Adopt Multi-Layer Side-Output (MLSO) training for deep supervision and layer-wise loss balancing.
• Integrate the learned metric into an FCN-based segmentation framework for end-to-end change detection.

Experimental results

Research questions

• RQ1 How to define and learn a discriminative metric that emphasizes semantic changes while suppressing noisy changes?
• RQ2 Can a siamese FCN with metric learning improve robustness to illumination and viewpoint variations in scene change detection?
• RQ3 Does Thresholded Contrastive Loss (TCL) provide tangible benefits under large viewpoint differences?
• RQ4 Does incorporating a distance metric into FCN-based change detection improve segmentation quality across datasets?

Key findings

• CosimNet with TCL outperforms baselines on VL-CMU-CD and PCD2015 datasets, with notable gains from using L2 distance and deeper, more semantic features.
• Original contrastive loss suffices for small viewpoint differences, while TCL improves performance under large viewpoint differences (PTZ scenarios).
• Euclidean distance generally yields higher RMS contrast and better change maps than cosine similarity across layers.
• MLSO training enhances intermediate-layer discriminativeness, improving overall change detection performance.
• CosimNet achieves competitive results on CDnet, particularly under large viewpoint differences, though semantic segmentation baselines remain strong in some metrics.