[Paper Review] Blind Image Quality Assessment for MRI with A Deep Three-dimensional content-adaptive Hyper-Network

Kehan Qi, Haoran Li+5 more|arXiv (Cornell University)|Jul 13, 2021

Image and Video Quality Assessment35 references4 citations

TL;DR

This paper proposes HyS-Net, a 3D content-adaptive hyper-network for blind MRI image quality assessment that leverages 3D spatial features and dynamically generated network parameters to improve performance. By integrating a 3D backbone with a content-aware hyper-network, the method achieves state-of-the-art SROCC (0.7566) and PLCC (0.8231) on the MRIQC dataset, outperforming existing methods by a significant margin.

ABSTRACT

Image Quality Assessment (IQA) is of great value in the workflow of Magnetic Resonance Imaging (MRI)-based analysis. Blind IQA (BIQA) methods are especially required since high-quality reference MRI images are usually not available. Recently, many efforts have been devoted to developing deep learning-based BIQA approaches. However, the performance of these methods is limited due to the utilization of simple content-non-adaptive network parameters and the waste of the important 3D spatial information of the medical images. To address these issues, we design a 3D content-adaptive hyper-network for MRI BIQA. The overall 3D configuration enables the exploration of comprehensive 3D spatial information from MRI images, while the developed content-adaptive hyper-network contributes to the self-adaptive capacity of network parameters and thus, facilitates better BIQA performance. The effectiveness of the proposed method is extensively evaluated on the open dataset, MRIQC. Promising performance is achieved compared with the corresponding baseline and 4 state-of-the-art BIQA methods. We make our code available at \url{https://git.openi.org.cn/SIAT_Wangshanshan/HyS-Net}.

Motivation & Objective

• To address the limitations of existing blind image quality assessment (BIQA) methods in MRI, particularly the lack of 3D spatial information exploitation and non-adaptive network parameters.
• To bridge the gap between model predictions and human perception by introducing content-adaptive network parameters that vary with image content.
• To develop a deep learning-based BIQA framework that effectively utilizes 3D spatial structure in MRI volumes for improved quality assessment.
• To evaluate the proposed method on a large-scale open MRI dataset and demonstrate its superiority over state-of-the-art approaches.

Proposed method

• A 3D convolutional neural network backbone is used to extract comprehensive spatial features from 3D MRI volumes, preserving volumetric context.
• A content-adaptive hyper-network generates dynamic, task-specific network weights based on input image content, enabling self-adaptive parameter tuning.
• The hyper-network takes a global context embedding from the MRI volume as input and produces the weights for a regression head, ensuring content-aware adaptation.
• The model is trained end-to-end using a regression loss to predict image quality scores from distorted MRI scans without reference images.
• The use of 3D spatial feature extraction ensures that the method captures inter-slice and intra-volume relationships crucial for MRI quality assessment.
• Principal component analysis (PCA) of generated weights confirms the content-adaptive nature of the hyper-network, showing distinct weight patterns for different image contents.

Experimental results

Research questions

• RQ1 Can a 3D spatial-aware architecture improve blind MRI image quality assessment compared to 2D approaches that ignore volumetric context?
• RQ2 Does a content-adaptive hyper-network that generates dynamic parameters based on image content lead to better alignment with human perception in BIQA?
• RQ3 How do the integration of 3D spatial features and content-adaptive weight generation jointly affect BIQA performance?
• RQ4 To what extent does the proposed method outperform existing state-of-the-art BIQA models on real MRI data?

Key findings

• HyS-Net achieves a Spearman’s rank-order correlation coefficient (SROCC) of 0.7566 and a Pearson’s linear correlation coefficient (PLCC) of 0.8231 on the MRIQC dataset, outperforming all compared methods.
• The ablation study confirms that combining the 3D spatial feature extractor with the content-adaptive hyper-network yields the highest performance gain, with SROCC increasing by 0.0243 compared to baseline.
• Introducing the content-adaptive hyper-network alone without 3D spatial features leads to marginal or even reduced performance, indicating that 3D context is essential for effective adaptation.
• The visualization of generated weights via PCA demonstrates that the hyper-network produces distinct, content-dependent weight patterns, validating its content-adaptive capability.
• The model with ResNet-50 backbone and both 3D spatial features and hyper-network performs slightly better than ResNet-18, but deeper networks alone degrade performance without adaptive components.
• Compared to four state-of-the-art methods—BRISQUE, DBCNN, deepIQA, and Two Stream CNN—HyS-Net achieves a 0.0307 higher SROCC and a 0.0234 higher PLCC, confirming its superior performance.