[论文解读] R$^2$-Gaussian: Rectifying Radiative Gaussian Splatting for Tomographic Reconstruction

Ruyi Zha, Lin Tao+4 more|arXiv (Cornell University)|May 31, 2024

Medical Imaging Techniques and ApplicationsMedicine被引用 3

一句话总结

R²-Gaussian 提出了一种基于3D高斯溅射的首个稀疏视角断层扫描重建方法，通过定制化的X射线光栅化和可微分CUDA体素化器，修正了标准3DGS中被忽视的积分偏差。该方法在38.90 dB PSNR下实现最先进性能，推理速度比NeRF-based方法快12倍，且在9分钟内完成收敛。

ABSTRACT

3D Gaussian splatting (3DGS) has shown promising results in image rendering and surface reconstruction. However, its potential in volumetric reconstruction tasks, such as X-ray computed tomography, remains under-explored. This paper introduces R$^2$-Gaussian, the first 3DGS-based framework for sparse-view tomographic reconstruction. By carefully deriving X-ray rasterization functions, we discover a previously unknown integration bias in the standard 3DGS formulation, which hampers accurate volume retrieval. To address this issue, we propose a novel rectification technique via refactoring the projection from 3D to 2D Gaussians. Our new method presents three key innovations: (1) introducing tailored Gaussian kernels, (2) extending rasterization to X-ray imaging, and (3) developing a CUDA-based differentiable voxelizer. Experiments on synthetic and real-world datasets demonstrate that our method outperforms state-of-the-art approaches in accuracy and efficiency. Crucially, it delivers high-quality results in 4 minutes, which is 12$ imes$ faster than NeRF-based methods and on par with traditional algorithms. Code and models are available on the project page https://github.com/Ruyi-Zha/r2_gaussian.

研究动机与目标

• 为解决稀疏视角X射线CT中有效3D高斯溅射（3DGS）方法的缺失问题。
• 识别并解决标准3DGS中此前未知的积分偏差，该偏差会降低断层扫描中密度场恢复的准确性。
• 开发一种可微分、专用于X射线的光栅化流水线，实现3D高斯在断层扫描重建中的端到端优化。
• 在极短训练时间内实现高质量重建，超越基于NeRF和传统CT的方法。

提出的方法

• 通过重构3DGS中的3D到2D投影过程，提出一种新颖的校正技术，以纠正影响X射线衰减估计的积分偏差。
• 设计专用于X射线成像的高斯核，提升沿投影路径的射线积分保真度。
• 扩展3DGS光栅化过程以处理X射线投影几何，实现通过3D高斯的可微分衰减值渲染。
• 开发基于CUDA的可微分体素化器，实现从3D高斯分布高效、可微分的体积分重建。
• 采用FDK初始化提供粗略初始结构，加速收敛并提升最终重建质量。
• 应用总变差（TV）正则化以抑制伪影并促进重建密度场的平滑性。

Figure 1: We compare our method to state-of-the-art NeRF-based methods (IntraTomo [ 61 ] , NAF [ 62 ] , SAX-NeRF [ 7 ] ) in terms of visual quality, PSNR (dB), and training time (minute). Our method achieves the highest reconstruction quality and is significantly faster than other methods. \cref@con

实验结果

研究问题

• RQ1 在应用于断层扫描重建时，标准3D高斯溅射中存在何种积分偏差？其对密度场恢复有何影响？
• RQ2 3D高斯溅射能否被有效适配于稀疏视角X射线计算机断层扫描，实现高保真度与快速收敛？
• RQ3 定制化的高斯核与专用于X射线的光栅化方法相比标准3DGS，如何提升重建精度？
• RQ4 所提出的可微分体素化器与优化策略在多大程度上实现了比基于NeRF的方法更快、更高质量的断层扫描重建？

主要发现

• R²-Gaussian 在测试集上实现38.90 dB PSNR，相比最先进基于NeRF的方法提升0.93 dB。
• 训练时间缩短至9分钟以内（具体为8分14秒，使用50k个高斯分布），比基于NeRF的方法快12倍。
• 完整模型相比基线方法将PSNR提升1.51 dB，SSIM提升0.018，其中FDK初始化贡献了0.9 dB增益。
• 最优性能在50k个初始化高斯分布、TV损失权重λ_tv = 0.05以及体素大小D = 32时达到。
• 收敛性分析表明，R²-Gaussian 在第500次迭代时即达到清晰、无伪影的结果，显著优于基于NeRF的方法。
• 该方法生成视图无关的重建结果，而X-3DGS则存在视图相关的伪影。

Figure 2: A detection plane captures the attenuation of X-rays emitted from different angles. \cref@constructprefix page \cref@result