[论文解读] DiffTrans: Differentiable Geometry-Materials Decomposition for Reconstructing Transparent Objects

Changpu Li, Shuang Wu+4 more|arXiv (Cornell University)|Feb 28, 2026

Computer Graphics and Visualization Techniques被引用 0

一句话总结

DiffTrans 引入一个可微分框架，利用多视图图像联合重建透明对象的几何和材料，采用 FlexiCubes 作为初始几何、环境辐射场，以及递归可微分网格光线追踪器来端到端优化折射率和吸收率。

ABSTRACT

Reconstructing transparent objects from a set of multi-view images is a challenging task due to the complicated nature and indeterminate behavior of light propagation. Typical methods are primarily tailored to specific scenarios, such as objects following a uniform topology, exhibiting ideal transparency and surface specular reflections, or with only surface materials, which substantially constrains their practical applicability in real-world settings. In this work, we propose a differentiable rendering framework for transparent objects, dubbed DiffTrans, which allows for efficient decomposition and reconstruction of the geometry and materials of transparent objects, thereby reconstructing transparent objects accurately in intricate scenes with diverse topology and complex texture. Specifically, we first utilize FlexiCubes with dilation and smoothness regularization as the iso-surface representation to reconstruct an initial geometry efficiently from the multi-view object silhouette. Meanwhile, we employ the environment light radiance field to recover the environment of the scene. Then we devise a recursive differentiable ray tracer to further optimize the geometry, index of refraction and absorption rate simultaneously in a unified and end-to-end manner, leading to high-quality reconstruction of transparent objects in intricate scenes. A prominent advantage of the designed ray tracer is that it can be implemented in CUDA, enabling a significantly reduced computational cost. Extensive experiments on multiple benchmarks demonstrate the superior reconstruction performance of our DiffTrans compared with other methods, especially in intricate scenes involving transparent objects with diverse topology and complex texture. The code is available at https://github.com/lcp29/DiffTrans.

研究动机与目标

• 从多视轮廓重建具有多样拓扑的透明对象几何。
• 同时恢复环境光照和场景辐射。
• 在端到端可微分框架中联合优化几何、折射率与吸收率。
• 实现具复杂纹理的透明对象场景编辑与再照明。

提出的方法

• 以 dilation 与平滑正则化的 ISO-曲面表示法使用 FlexiCubes 以从多视掩模获取初始几何。
• 从掩膜外区域初始化环境光辐射场。
• 开发一个在 CUDA/OptiX 实现的递归可微分网格光线追踪器，以联合优化几何、IoR 与吸收率。
• 通过可微分的 3D 吸收纹理和简化的辐射传输方程来建模吸收介质中的光传输。
• 应用带场景正则化损失的可微分网格-光线追踪以细化几何与材料属性。

实验结果

研究问题

• RQ1 透明对象在复杂拓扑下是否能够从多视图图像联合重建几何与材料（IoR 与吸收）？
• RQ2 相较于仅关注表面或不透明目标的先前方法，光线追踪中的可微分递归如何提升几何与材料估计？
• RQ3 初始几何与环境初始化对收敛性与最终重建质量有何影响？
• RQ4 同时建模吸收与屈折对再照明质量与新视图合成有何影响？
• RQ5 在透明对象的可微 rendering 中，初始化稳定性与端到端优化之间有哪些权衡？

主要发现

Method	CD(×10^-4) ↓	F1(×10^-1) ↑
NeRO	36.022	5.691
NU-NeRF	7.891	8.026
NeRRF	13.341	6.916
Ours(S1)	4.666	8.088
Ours	3.264	8.386

• DiffTrans 在合成场景中的几何重建平均表现优越（CD 更低）且 F1 得分更高，相较 NeRO、NU-NeRF 与 NeRRF。
• 六个合成场景中的学习 IoR 值与真值非常接近，存在较小差异。
• 与递归可微分网格光线追踪器的联合优化比初始阶段提升了几何细化。
• DiffTrans 实现了合理的再照明结果，在 PSNR、SSIM 与 LPIPS 指标上优于基线方法。
• 色调正则化在新视图合成与再照明性能方面对各场景均有总体提升。