[Paper Review] AE-Net: Autonomous Evolution Image Fusion Method Inspired by Human Cognitive Mechanism

Aiqing Fang, Xinbo Zhao+3 more|arXiv (Cornell University)|Jul 17, 2020

Advanced Image Fusion Techniques58 references4 citations

TL;DR

AE-Net proposes a novel image fusion method inspired by human brain cognitive mechanisms, enabling autonomous evolution through multi-method collaboration, multi-index evaluation, and iterative learning. It transforms unsupervised image fusion into a semi-supervised task, significantly improving robustness and generalization across diverse datasets including infrared-visible, multi-focus, and medical imaging.

ABSTRACT

In order to solve the robustness and generality problems of the image fusion task,inspired by the human brain cognitive mechanism, we propose a robust and general image fusion method with autonomous evolution ability, and is therefore denoted with AE-Net. Through the collaborative optimization of multiple image fusion methods to simulate the cognitive process of human brain, unsupervised learning image fusion task can be transformed into semi-supervised image fusion task or supervised image fusion task, thus promoting the evolutionary ability of network model weight. Firstly, the relationship between human brain cognitive mechanism and image fusion task is analyzed and a physical model is established to simulate human brain cognitive mechanism. Secondly, we analyze existing image fusion methods and image fusion loss functions, select the image fusion method with complementary features to construct the algorithm module, establish the multi-loss joint evaluation function to obtain the optimal solution of algorithm module. The optimal solution of each image is used to guide the weight training of network model. Our image fusion method can effectively unify the cross-modal image fusion task and the same modal image fusion task, and effectively overcome the difference of data distribution between different datasets. Finally, extensive numerical results verify the effectiveness and superiority of our method on a variety of image fusion datasets, including multi-focus dataset, infrared and visi-ble dataset, medical image dataset and multi-exposure dataset. Comprehensive experiments demonstrate the superiority of our image fusion method in robustness and generality. In addition, experimental results also demonstate the effectiveness of human brain cognitive mechanism to improve the robustness and generality of image fusion.

Motivation & Objective

• Address the lack of robustness and generalization in existing image fusion methods under complex, varying data distributions.
• Overcome the limitations of unsupervised learning in image fusion due to the absence of reliable labels and quality metrics.
• Introduce a continuous learning mechanism inspired by human brain cognitive processes to enable autonomous model evolution.
• Unify cross-modal and same-modal image fusion tasks under a single, adaptive framework.
• Enhance the generalization capability of deep learning-based image fusion by integrating knowledge synergy from multiple complementary methods.

Proposed method

• Proposes a three-module framework: multi-method collaborative module, multi-index evaluation module, and iterative learning optimization module.
• Simulates human brain cognitive mechanisms via a physical model that supports working memory and continuous learning.
• Uses a multi-loss joint evaluation function to assess fusion quality across multiple image quality indices, selecting optimal solutions per image.
• Transforms unsupervised image fusion into a semi-supervised task by using the optimal fusion results from multiple methods as pseudo-labels.
• Employs iterative optimization to update network weights based on the best-performing fusion results, enabling autonomous evolution.
• Leverages complementary features from diverse image fusion methods to enhance feature selection and nonlinear fusion characteristics.

Experimental results

Research questions

• RQ1 Can simulating human brain cognitive mechanisms improve the robustness and generalization of image fusion networks?
• RQ2 How can unsupervised image fusion be effectively transformed into a semi-supervised learning problem?
• RQ3 To what extent does knowledge synergy among multiple image fusion methods enhance performance and stability?
• RQ4 Can a multi-loss evaluation function effectively guide the evolution of network weights toward optimal fusion results?
• RQ5 How does the autonomous evolution mechanism perform across diverse image fusion tasks with varying data distributions?

Key findings

• AE-Net achieves superior performance across five diverse image fusion tasks, including multi-focus, infrared-visible, multi-exposure, and medical imaging.
• The method demonstrates significant robustness to distribution shifts between datasets, outperforming state-of-the-art methods in cross-dataset generalization.
• Comprehensive ablation studies confirm that the integration of cognitive mechanism simulation enhances model stability and learning efficiency.
• The use of multi-index evaluation and pseudo-labeling from multiple fusion methods improves convergence and final fusion quality.
• AE-Net’s autonomous evolution mechanism enables continuous improvement without retraining from scratch, mimicking human brain learning patterns.
• Experimental results validate that human brain-inspired cognitive mechanisms significantly enhance robustness and generalization in image fusion.