[论文解读] Gradient Aware Cascade Network for Multi-Focus Image Fusion.

Boyuan Ma, Xiang Yin+3 more|arXiv (Cornell University)|Oct 17, 2020

Advanced Image Fusion Techniques被引用 4

一句话总结

本文提出了一种梯度感知级联网络用于端到端的多焦点图像融合，联合生成决策图与融合图像，消除了对经验后处理的依赖。通过引入梯度感知损失函数与决策校准机制，该方法在融合质量方面有所提升，并且在多幅图像融合任务中相比最先进方法速度提升30%以上。

ABSTRACT

The general aim of multi-focus image fusion is to gather focused regions of different images to generate a unique all-in-focus fused image. Deep learning based methods become the mainstream of image fusion by virtue of its powerful feature representation ability. However, most of the existing deep learning structures failed to balance fusion quality and end-to-end implementation convenience. End-to-end decoder design often leads to poor performance because of its non-linear mapping mechanism. On the other hand, generating an intermediate decision map achieves better quality for the fused image, but relies on the rectification with empirical post-processing parameter choices. In this work, to handle the requirements of both output image quality and comprehensive simplicity of structure implementation, we propose a cascade network to simultaneously generate decision map and fused result with an end-to-end training procedure. It avoids the dependence on empirical post-processing methods in the inference stage. To improve the fusion quality, we introduce a gradient aware loss function to preserve gradient information in output fused image. In addition, we design a decision calibration strategy to decrease the time consumption in the application of multiple image fusion. Extensive experiments are conducted to compare with 16 different state-of-the-art multi-focus image fusion structures with 6 assessment metrics. The results prove that our designed structure can generally ameliorate the output fused image quality, while implementation efficiency increases over 30\% for multiple image fusion.

研究动机与目标

• 解决基于深度学习的多焦点图像融合中融合质量与结构简洁性之间的权衡问题。
• 通过实现决策图与融合图像生成的端到端训练，消除对经验后处理的依赖。
• 通过一种新型损失函数提升梯度保留能力，从而增强融合质量。
• 通过决策校准策略降低多幅图像融合中的推理时间。

提出的方法

• 设计了一种级联网络架构，实现决策图与融合图像的端到端联合预测。
• 引入一种梯度感知损失函数，通过在融合输出中保持梯度信息，以保留结构与边缘细节。
• 提出一种决策校准策略，以降低推理过程中的计算开销，尤其在多幅图像融合场景中效果显著。
• 网络采用端到端训练，无需手动调整后处理参数。
• 该方法利用深层特征表示提升融合性能，同时保持结构简洁性。

实验结果

研究问题

• RQ1 统一的端到端网络架构是否能够无需后处理即可联合生成决策图与融合图像？
• RQ2 梯度感知损失如何提升融合图像的感知质量与结构质量？
• RQ3 决策校准在多幅图像融合应用中能在多大程度上减少推理时间？
• RQ4 与16种最先进融合网络相比，所提方法在性能与效率方面表现如何？

主要发现

• 与16种最先进方法相比，所提方法在六项评估指标上均实现了更优的融合质量。
• 网络消除了对经验后处理的依赖，提升了推理的简洁性与可复现性。
• 梯度感知损失函数有效保留了边缘与纹理细节，提升了融合图像的结构相似性与清晰度。
• 决策校准策略使多幅图像融合的时间消耗减少了30%以上。