[论文解读] EDiffSR: An Efficient Diffusion Probabilistic Model for Remote Sensing Image Super-Resolution

Yi Xiao, Qiangqiang Yuan+4 more|arXiv (Cornell University)|Oct 30, 2023

Advanced Image Fusion Techniques被引用 8

一句话总结

EDiffSR 引入一种轻量级扩散概率模型，用于遥感图像超分辨率，结合条件先验增强模块和高效激活网络，在比以往基于DPM的SR方法更少的参数下提升感知质量。

ABSTRACT

Recently, convolutional networks have achieved remarkable development in remote sensing image Super-Resoltuion (SR) by minimizing the regression objectives, e.g., MSE loss. However, despite achieving impressive performance, these methods often suffer from poor visual quality with over-smooth issues. Generative adversarial networks have the potential to infer intricate details, but they are easy to collapse, resulting in undesirable artifacts. To mitigate these issues, in this paper, we first introduce Diffusion Probabilistic Model (DPM) for efficient remote sensing image SR, dubbed EDiffSR. EDiffSR is easy to train and maintains the merits of DPM in generating perceptual-pleasant images. Specifically, different from previous works using heavy UNet for noise prediction, we develop an Efficient Activation Network (EANet) to achieve favorable noise prediction performance by simplified channel attention and simple gate operation, which dramatically reduces the computational budget. Moreover, to introduce more valuable prior knowledge into the proposed EDiffSR, a practical Conditional Prior Enhancement Module (CPEM) is developed to help extract an enriched condition. Unlike most DPM-based SR models that directly generate conditions by amplifying LR images, the proposed CPEM helps to retain more informative cues for accurate SR. Extensive experiments on four remote sensing datasets demonstrate that EDiffSR can restore visual-pleasant images on simulated and real-world remote sensing images, both quantitatively and qualitatively. The code of EDiffSR will be available at https://github.com/XY-boy/EDiffSR

研究动机与目标

• 推动在 RSI SR 中提升感知质量，超越基于MSE的回归以减少过度平滑。
• 开发适用于大规模 RSI 应用的轻量级扩散式超分框架。
• 将来自低分辨率影像的有信息先验引入以引导扩散式超分。
• 提出高效去噪网络以在保持质量的同时降低计算负担。

提出的方法

• 对 SR 采用均值回归的随机微分方程（SDE）扩散过程。
• 引入条件先验增强模块（CPEM），用低分辨率先验丰富 SR 条件信息。
• 设计带有高效激活块（EAB）的高效激活网络（EANet）用于噪声预测。
• 以最大似然类目标进行训练，稳定扩散训练过程。
• 使用像素折叠操作在对模型进行条件化前对齐噪声与LR条件输入。

Figure 1: The relationship between FID (Fréchet Inception Distance) [ 1 ] performance and parameter of state-of-the-art (SOTA) SR methods (lower FID values indicate better generative quality). EDSR [ 2 ] , RCAN [ 3 ] , and HAT-L [ 4 ] are regression-based models, typically generating low-quality dis

实验结果

研究问题

• RQ1 在扩散框架中对 LR 基先验的丰富是否能比双三次插值条件或基于 Unet 的噪声预测在SR上获得更高保真度？
• RQ2 具有 EAB 的高效去噪器（EANet）是否在参数更少的情况下达到与最先进 DPM 相当的 SR 结果？
• RQ3 条件先验增强模块对遥感数据集上 SR 的感知与定量指标有何影响？

主要发现

Categories	Bicubic	EDSR	RCAN	HAT-L	MSRGAN	ESRGAN	SPSR	SR3	IRSDE	EDiffSR
Average	126.53	90.40	93.39	90.86	53.36	52.55	56.40	65.51	51.08	49.45

• EDiffSR 在多个 RSI 数据集上相比 CNN、GAN 与 DPM 基的 SR 方法取得更优的感知与定量性能。
• 提出的 CPEM 从 LR 输入中丰富了条件信息，改善高频细节的恢复。
• EANet 与 EAB 在显著降低计算预算的情况下提供有利的去噪能力，相较于大型 UNet。
• 在多个数据集上的结果表明对模拟降级和真实 RSI 的泛化能力较强。
• 平均来看，EDiffSR 在基于 FID 的指标上相较竞争方法有提升。

Figure 2: Overview of the forward and reverse diffusion processes defined by mean-reverting Stochastic Differential Equations (SDE). The forward diffusion gradually degrades the high-quality and high-resolution image $x_{0}$ to its low-quality counterpart via $x_{T}=\mu+\varepsilon$ . The reverse di