[Paper Review] Weakly Supervised Change Detection in a Pair of Images.

Salman Khan, Xuming He+4 more|arXiv (Cornell University)|Jun 7, 2016

Remote-Sensing Image Classification3 citations

TL;DR

This paper proposes a weakly supervised deep learning framework for change detection in image pairs using only image-level labels, avoiding costly pixel-level annotations. It combines a CNN with a CRF and an iterative EM optimization to learn discriminative change patterns, achieving state-of-the-art performance on four benchmark datasets with improved localization and detection accuracy.

ABSTRACT

Conventional change detection methods require a large number of images to learn background models. The few recent approaches that attempt change detection between two images either use handcrafted features or depend strongly on tedious pixel-level labeling by humans. In this paper, we present a weakly supervised approach that needs only image-level labels to simultaneously detect and localize changes in a pair of images. To this end, we employ a deep neural network with DAG topology to { learn patterns of change} from image-level labeled training data. On top of the initial CNN activations, we define a CRF model to incorporate the local differences and the dense connections between individual pixels. We apply a constrained mean-field algorithm to estimate the pixel-level labels, and use the estimated labels to update the parameters of the CNN in an iterative EM framework. This enables imposing global constraints on the observed foreground probability mass function. Our evaluations on four large benchmark datasets demonstrate superior detection and localization performance.

Motivation & Objective

• To address the limitation of conventional change detection methods that require large amounts of pixel-level annotated data.
• To enable accurate change detection and localization using only image-level labels, reducing human annotation effort.
• To develop a deep learning framework that learns change patterns from weakly supervised signals.
• To incorporate global constraints on foreground probability through iterative parameter updates in an EM framework.
• To achieve superior performance on benchmark datasets without relying on expensive pixel-level annotations.

Proposed method

• A deep neural network with DAG topology is used to learn change patterns from image-level labeled data.
• A CRF model is applied on top of CNN activations to capture local differences and dense pixel-level dependencies.
• A constrained mean-field inference algorithm estimates pixel-level labels from the CRF model.
• An iterative EM framework alternates between estimating pixel-level labels and updating CNN parameters to improve feature learning.
• Global constraints are imposed on the foreground probability mass function during optimization to enhance localization accuracy.
• The method jointly optimizes feature learning and label estimation using weak supervision from image-level annotations.

Experimental results

Research questions

• RQ1 Can change detection be effectively performed using only image-level labels instead of pixel-level annotations?
• RQ2 How can local pixel-level differences be modeled effectively within a weakly supervised deep learning framework?
• RQ3 Can global constraints on the foreground probability distribution improve change localization?
• RQ4 Does the integration of CRF with CNN and EM optimization lead to better performance than existing weakly supervised methods?
• RQ5 How does the proposed method generalize across diverse change detection benchmarks?

Key findings

• The proposed method achieves state-of-the-art performance on four large-scale benchmark datasets for change detection.
• It significantly reduces the need for expensive pixel-level annotations by relying solely on image-level labels.
• The integration of CRF with CNN and EM optimization improves localization accuracy by modeling dense pixel-level relationships.
• The iterative EM framework effectively refines both feature learning and label estimation, enhancing overall detection performance.
• The method demonstrates robust generalization across diverse image pairs and change scenarios in benchmark evaluations.
• The use of global constraints on the foreground probability mass function leads to more coherent and accurate change maps.