[论文解读] GRAD-Former: Gated Robust Attention-based Differential Transformer for Change Detection

Durgesh Ameta, Ujjwal Mishra+2 more|arXiv (Cornell University)|Mar 1, 2026

Remote-Sensing Image Classification被引用 0

一句话总结

GRAD-Former 提出一种基于 Siamese 变换器的变化检测模型，具备 Adaptive Feature Relevance And Refinement (AFRAR) 和 differential attention，可高效检测超高分辨率遥感图像中的变化，在参数更少的情况下实现业界领先结果。

ABSTRACT

Change detection (CD) in remote sensing aims to identify semantic differences between satellite images captured at different times. While deep learning has significantly advanced this field, existing approaches based on convolutional neural networks (CNNs), transformers and Selective State Space Models (SSMs) still struggle to precisely delineate change regions. In particular, traditional transformer-based methods suffer from quadratic computational complexity when applied to very high-resolution (VHR) satellite images and often perform poorly with limited training data, leading to under-utilization of the rich spatial information available in VHR imagery. We present GRAD-Former, a novel framework that enhances contextual understanding while maintaining efficiency through reduced model size. The proposed framework consists of a novel encoder with Adaptive Feature Relevance and Refinement (AFRAR) module, fusion and decoder blocks. AFRAR integrates global-local contextual awareness through two proposed components: the Selective Embedding Amplification (SEA) module and the Global-Local Feature Refinement (GLFR) module. SEA and GLFR leverage gating mechanisms and differential attention, respectively, which generates multiple softmax heaps to capture important features while minimizing the captured irreverent features. Multiple experiments across three challenging CD datasets (LEVIR-CD, CDD, DSIFN-CD) demonstrate GRAD-Former's superior performance compared to existing approaches. Notably, GRAD-Former outperforms the current state-of-the-art models across all the metrics and all the datasets while using fewer parameters. Our framework establishes a new benchmark for remote sensing change detection performance. Our code will be released at: https://github.com/Ujjwal238/GRAD-Former

研究动机与目标

• 通过解决现有 CNN/Transformer/SSM 方法中噪声和特征使用低效的问题，推动 VHR 遥感中的鲁棒变化检测（CD）。
• 提出带 AFRAR（SEA 与 GLFR）的 GRAD-Former，以过滤噪声并捕捉全局-局部上下文。
• 引入 Differential Amalgamation (DA) 将来自前后时相的语义特征与差分特征融合。
• 在多个公开数据集上展示更少参数情况下的最先进 CD 性能。

提出的方法

• 提出一个带编码器、融合模块和解码器的 Siamese 变换器框架用于 CD。
• 引入 AFRAR 模块，将特征分成 SEA 与 GLFR 分支以实现选择性放大和差分注意。
• SEA 使用带 L2 归一化的门控嵌入及可学习参数来放大相关通道。
• GLFR 使用差分多头注意，通过将两个 softmax 堆叠并结合一个可学习的标量缩放，创建稀疏、抗噪声的注意力图。
• Differential Amalgamation (DA) 将前变、后变及其差分串联起来，随后用 1x1 卷积进行特征融合。
• 解码器聚合多阶段融合特征，使用转置卷积进行上采样，包含残差块，并输出二值变化图。

实验结果

研究问题

• RQ1 AFRAR 是否能够在 VHR CD 中有效过滤噪声和无关信息，从而改善变化区域的勾勒？
• RQ2 GLFR 中的差分注意是否在降低计算开销的同时提升全局-局部上下文建模？
• RQ3 整体 GRAD-Former 架构在标准基准上是否以更少的参数获得更优的 CD 精度？

主要发现

Type	Method	Publication	CDD F1	CDD IoU	CDD OA	DSIFN-CD F1	DSIFN-CD IoU	DSIFN-CD OA	LEVIR-CD F1	LEVIR-CD IoU	LEVIR-CD OA
Transformer-based	GRAD-Former	-	97.57	95.26	99.43	93.14	87.16	97.65	91.52	84.36	99.14

• GRAD-Former 在 LEVIR-CD、DSIFN-CD 和 CDD 数据集上达到最先进的性能。
• 在所有报道的指标上，该模型都优于现有方法，同时参数更少。
• 消融分析表明 AFRAR（SEA 与 GLFR）和 DA 模块对性能提升具有贡献。