[Paper Review] Advanced Feature Manipulation for Enhanced Change Detection Leveraging Natural Language Models

Zhenglin Li, Yangchen Huang+4 more|arXiv (Cornell University)|Mar 23, 2024

Customer churn and segmentation22 citations

TL;DR

The paper introduces feature manipulation techniques—Feature Attention and Flow Dual-Alignment Fusion (FDAF)—to enhance DDPM-based change detection, achieving state-of-the-art metrics on LEVIR-CD and strong results on WHU-CD. It also analyzes the impact of these modules and discusses limitations of FDAF.

ABSTRACT

Change detection is a fundamental task in computer vision that processes a bi-temporal image pair to differentiate between semantically altered and unaltered regions. Large language models (LLMs) have been utilized in various domains for their exceptional feature extraction capabilities and have shown promise in numerous downstream applications. In this study, we harness the power of a pre-trained LLM, extracting feature maps from extensive datasets, and employ an auxiliary network to detect changes. Unlike existing LLM-based change detection methods that solely focus on deriving high-quality feature maps, our approach emphasizes the manipulation of these feature maps to enhance semantic relevance.

Motivation & Objective

• Motivate improved change detection in bi-temporal remote sensing images beyond plain diffusion-based feature extraction.
• Propose semantic feature manipulation to exploit temporal information via attention mechanisms.
• Introduce and evaluate attention-based blocks (Siam-SR and Cot-SR) and the Flow Dual-Alignment Fusion (FDAF) strategy.
• Assess performance on LEVIR-CD and WHU-CD datasets and analyze robustness to ambient noise.

Proposed method

• Use a pre-trained diffusion model to extract feature maps from bi-temporal images.
• Introduce Siam-SR (Siamese Semantic Reasoning) to encode per-timepoint semantic information.
• Introduce Cot-SR (Cross-temporal Semantic Reasoning) to model temporal correlations across time points.
• Propose FDAF (Flow Dual-Alignment Fusion) to align and fuse dual diffusion features with FlowNet and image warping.
• Keep diffusion model parameters fixed during training to isolate effects of feature manipulation.
• Train with BCE loss and AdamW optimizer; learning rate starts at 1e-5 with linear decay; 120 epochs (LEVIR-CD) and 80 epochs (WHU-CD).

(a) Attention Based Model.

Experimental results

Research questions

• RQ1 Can semantic attribution within each temporal branch improve change detection beyond standard diffusion-based features?
• RQ2 Do cross-temporal semantic reasoning blocks (Cot-SR) improve detection of changes over time compared to per-frame semantic encoding?
• RQ3 Does FDAF improve robustness to environmental noise in bi-temporal imagery, and under what conditions does it help or hurt performance?
• RQ4 What is the empirical performance of the proposed feature manipulation modules on LEVIR-CD and WHU-CD datasets?

Key findings

• Feature Attention yields state-of-the-art F1 and IoU on LEVIR-CD (F1 90.18, IoU 83.86).
• On LEVIR-CD, Baseline, Baseline+Attention, Baseline+FDAF, and Baseline+Attention+FDAF achieve: 90.91/83.35, 91.52/84.36, 82.61/70.37, and 91.37/84.11 respectively.
• On WHU-CD, Baseline, Baseline+Attention, Baseline+FDAF, and Baseline+Attention+FDAF achieve: 92.65/86.31, 92.40/85.87, 76.33/61.71, and 90.27/82.27 respectively.
• Feature Attention consistently improves LEVIR-CD performance and demonstrates the value of correlational learning in temporal feature maps.
• FDAF shows dataset-specific performance dynamics: beneficial in some setups but can degrade results if noise filtering removes key feature differences.

(b) FDAF Based Model