[论文解读] High-quality Image Dehazing with Diffusion Model

Hu Yu, Jie Huang+2 more|arXiv (Cornell University)|Aug 23, 2023

Image Enhancement Techniques被引用 13

一句话总结

该论文提出 DehazeDDPM，这是一个两阶段、物理感知的基于扩散的框架，将大气散射模型（ASM）與条件 DDPM 相结合，以实现最先进的去雾，尤其在密集雾场情景。

ABSTRACT

Image dehazing is quite challenging in dense-haze scenarios, where quite less original information remains in the hazy image. Though previous methods have made marvelous progress, they still suffer from information loss in content and color in dense-haze scenarios. The recently emerged Denoising Diffusion Probabilistic Model (DDPM) exhibits strong generation ability, showing potential for solving this problem. However, DDPM fails to consider the physics property of dehazing task, limiting its information completion capacity. In this work, we propose DehazeDDPM: A DDPM-based and physics-aware image dehazing framework that applies to complex hazy scenarios. Specifically, DehazeDDPM works in two stages. The former stage physically models the dehazing task with the Atmospheric Scattering Model (ASM), pulling the distribution closer to the clear data and endowing DehazeDDPM with fog-aware ability. The latter stage exploits the strong generation ability of DDPM to compensate for the haze-induced huge information loss, by working in conjunction with the physical modelling. Extensive experiments demonstrate that our method attains state-of-the-art performance on both synthetic and real-world hazy datasets.

研究动机与目标

• 通过利用物理信息的两阶段方法，解决在密集雾下内容与颜色的信息损失。
• 通过受物理建模引导的条件 DDPM 来记住清晰图像的分布。
• 通过雾的感知和分布-更近条件，在现实世界的复杂雾霾场景中改进去雾。
• 在扩散过程通过频率先验增强高频细节恢复。

提出的方法

• 阶段1：基于物理的网络在大气散射模型（ASM）的引导下估计传输图 trmap、无雾图像 J 和大气光照 A。
• 阶段2：一个条件 DDPM 使用 Fog-aware and Distribution-closer Conditions (FDC) 与 Confidence-guided Dynamic Fusion (CDF) 来向清晰分布进行细化。
• 训练目标包括标准 DDPM 损失以及一个频率先验损失，以强调高频恢复。
• 引入两阶段融合，其中第一阶段输出的 J 和 trmap 通过 trmap 加权融合（CDF）为第二阶段扩散过程提供信息。
• 在训练期间通过指数移动平均（EMA）来缓解颜色偏移。

Figure 1: The visual examples of dehazing results were sampled from real-world hazy images. The second to fourth columns show the results of Dehamer [ 3 ] , our first-stage, and our DehazeDDPM, respectively. Our method demonstrates unprecedented perceptual quality on the challenging real-world datas

实验结果

研究问题

• RQ1 如何通过在两阶段框架中整合 ASM 使扩散模型在去雾方面具备物理感知？
• RQ2 将 DDPM 条件化于去雾估计和估计的传输图是否在密集雾数据集上提升性能？
• RQ3 频率先验和动态融合是否能改善高频细节的恢复和感知质量？
• RQ4 所提出的 DehazeDDPM 在真实世界的雾霾场景中对比合成基准的泛化能力如何？

主要发现

方法	Dense-Haze FID	Dense-Haze LPIPS	Dense-Haze PSNR	Dense-Haze SSIM	NH-HAZE FID	NH-HAZE LPIPS	NH-HAZE PSNR	NH-HAZE SSIM	SOTS FID	SOTS LPIPS	SOTS PSNR	SOTS SSIM
DCP	343.65	0.6050	10.06	0.3856	348.36	0.3994	10.57	0.5196	30.85	0.0694	15.09	0.7649
AOD-Net	415.82	0.5991	13.14	0.4144	461.42	0.4947	15.40	0.5693	48.56	0.0988	19.82	0.8178
GridNet	429.73	0.5102	13.31	0.3681	331.15	0.3046	13.80	0.5370	3.93	0.0081	32.16	0.9836
FFA-Net	413.22	0.4976	14.39	0.4524	374.31	0.3653	19.87	0.6915	2.30	0.0048	36.39	0.9886
MSBDN	335.03	0.5358	15.37	0.4858	287.81	0.2918	19.23	0.7056	8.13	0.0288	33.79	0.9840
AECR-Net	335.18	0.5368	15.80	0.4660	196.38	0.2782	19.88	0.7073	3.18	0.0068	37.17	0.9901
Dehamer	223.65	0.4796	16.62	0.5602	138.49	0.2296	20.66	0.6844	2.54	0.0046	36.63	0.9881
FSDGN	202.94	0.4190	16.91	0.5806	177.63	0.2248	19.99	0.7106	1.54	0.0037	38.63	0.9903
RIDCP	316.38	0.5507	8.0857	0.4173	367.67	0.3578	12.27	0.4996	45.24	0.6506	18.3562	0.7526
First-stage	221.17	0.4272	15.86	0.5588	184.30	0.2376	18.73	0.6494	1.72	0.0043	37.88	0.9887
DehazeDDPM	171.06	0.2994	19.04	0.5922	121.06	0.1624	22.28	0.7309	1.68	0.0041	38.03	0.9895

• 在密集雾数据集（Dense-Haze 和 NH-HAZE）上达到最先进的性能，并在合成 SOTS 上有竞争力的结果。
• 与现有方法相比，在感知 (FID, LPIPS) 与失真 (PSNR, SSIM) 指标方面有显著提升。
• 第一阶段基于 ASM 的估计使雾霾数据分布更接近清晰数据，为 DDPM 提供雾感知的引导。
• FDC 与 CDF 能稳健处理易区和难区，提升实际雾霾场景的恢复保真度。
• 频率先验损失有助于强调高频细节，促使去雾输出更清晰。

Figure 2: Statistics illustration of the haze-induced information loss, including t-SNE clustering [ 14 ] , distribution distance, histogram, gradient, entropy, and standard deviation. The dense haze causes massive information loss in content and color.