[论文解读] Triggering hallucinations in model-based MRI reconstruction via adversarial perturbations

Suna Buğday, Yvan Saeys+1 more|arXiv (Cornell University)|Feb 20, 2026

Adversarial Robustness in Machine Learning被引用 0

一句话总结

本论文证明了基于模型的MRI重建（UNet与E2E-VarNet）在对不可察觉的对抗扰动下极易产生幻觉，这些幻觉难以通过标准指标检测。

ABSTRACT

Generative models are increasingly used to improve the quality of medical imaging, such as reconstruction of magnetic resonance images and computed tomography. However, it is well-known that such models are susceptible to hallucinations: they may insert features into the reconstructed image which are not actually present in the original image. In a medical setting, such hallucinations may endanger patient health as they can lead to incorrect diagnoses. In this work, we aim to quantify the extent to which state-of-the-art generative models suffer from hallucinations in the context of magnetic resonance image reconstruction. Specifically, we craft adversarial perturbations resembling random noise for the unprocessed input images which induce hallucinations when reconstructed using a generative model. We perform this evaluation on the brain and knee images from the fastMRI data set using UNet and end-to-end VarNet architectures to reconstruct the images. Our results show that these models are highly susceptible to small perturbations and can be easily coaxed into producing hallucinations. This fragility may partially explain why hallucinations occur in the first place and suggests that a carefully constructed adversarial training routine may reduce their prevalence. Moreover, these hallucinations cannot be reliably detected using traditional image quality metrics. Novel approaches will therefore need to be developed to detect when hallucinations have occurred.

研究动机与目标

• 量化使用神经网络进行的模型基MRI重建中幻觉发生的程度。
• 证明一种对抗攻击，通过不可察觉的输入扰动向重建中注入目标细节。
• 评估传统图像质量指标是否能检测到幻觉。
• 提供关于防御与检测方法的见解，以实现鲁棒的MRI重建。

提出的方法

• 对输入k-space数据实施看不见的扰动，以迫使重建在掩模区域内包含目标细节。
• 在L∞预算下通过迭代式类似BIM的过程优化扰动。
• 使用目标重建和掩模在注入幻觉与保持对原始重建忠实之间取得平衡。
• 在fastMRI脑部和膝部数据上使用UNet与E2E-VarNet重建评估扰动。
• 通过比较扰动前后输入与重建的PSNR、NRMSE与SSIM来衡量成功性。

(a)

实验结果

研究问题

• RQ1 不可察觉的扰动是否会在基于模型的MRI重建中引发幻觉？
• RQ2 幻觉是否可被标准图像质量指标（PSNR、NRMSE、SSIM）检测？
• RQ3 模型基MRI重建在小幅输入扰动下是否表现出不稳定性？
• RQ4 扰动是否可用于构建用于检测或缓解幻觉的数据集？

主要发现

• 对抗扰动在膝部和脑部数据的UNet与E2E-VarNet重建中产生幻觉。
• 扰动导致重建显著失真，而输入扰动在视觉上仍不可察觉。
• 扰动重建的PSNR、NRMSE与SSIM分布与干净重建大体分离，但仍有部分重叠。
• 在将扰动输入与重建进行比较时，标准指标较难检测到幻觉。
• 定性结果表明所插入的细节可能引发超出目标区域的更广泛畸变。
• 建议采用对抗训练或基于原理的防御来提升鲁棒性。

(a)