[论文解读] FreeArtGS: Articulated Gaussian Splatting Under Free-moving Scenario
FreeArtGS 在单目 RGB-D 视频中通过联合分割移动部件、估计关节类型/轴,并优化用于外观、几何和关节的高斯斯帕汀来重构自由移动的关节对象。它在 FreeArt-21、Video2Articulation-S 和真实世界数据上均优于基线方法。
The increasing demand for augmented reality and robotics is driving the need for articulated object reconstruction with high scalability. However, existing settings for reconstructing from discrete articulation states or casual monocular videos require non-trivial axis alignment or suffer from insufficient coverage, limiting their applicability. In this paper, we introduce FreeArtGS, a novel method for reconstructing articulated objects under free-moving scenario, a new setting with a simple setup and high scalability. FreeArtGS combines free-moving part segmentation with joint estimation and end-to-end optimization, taking only a monocular RGB-D video as input. By optimizing with the priors from off-the-shelf point-tracking and feature models, the free-moving part segmentation module identifies rigid parts from relative motion under unconstrained capture. The joint estimation module calibrates the unified object-to-camera poses and recovers joint type and axis robustly from part segmentation. Finally, 3DGS-based end-to-end optimization is implemented to jointly reconstruct visual textures, geometry, and joint angles of the articulated object. We conduct experiments on two benchmarks and real-world free-moving articulated objects. Experimental results demonstrate that FreeArtGS consistently excels in reconstructing free-moving articulated objects and remains highly competitive in previous reconstruction settings, proving itself a practical and effective solution for realistic asset generation. The project page is available at: https://freeartgs.github.io/
研究动机与目标
- 在没有静态基部件或固定关节状态的自由移动场景中,推动可扩展的关节对象重构。
- 提出一个三模块管线(自由移动部件分割、关节估计、端到端高斯斯帕汀优化)。
- 在新的 FreeArt-21 基准、Video2Articulation-S 设置和真实世界对象上展示鲁棒性和准确性。
提出的方法
- 使用密集级别的二维跟踪和预训练特征,进行基于优化的自由移动部件分割,且不依赖静态基部件假设。
- 使用现成的姿态估计器估计部件到相机的变换,以推导两个规范化部件高斯分布并将其校准到统一坐标系。
- 从相对部件运动推断关节类型(转动关节或直线关节)和轴;通过两阶段端到端优化,利用可微分高斯斯帕汀渲染器细化外观、几何、相机参数与关节信息。
实验结果
研究问题
- RQ1在自由移动的设定中,当对象位姿和关节状态任意变化时,是否能够实现对关节对象的准确重建?
- RQ2如何在不依赖静态基部件或预定义运动模式的情况下实现鲁棒的部件分割与关节估计?
- RQ3在单目 RGB-D 输入下,端到端优化结合高斯斯帕汀能否实现高保真几何、纹理与准确的关节构型?
主要发现
- FreeArtGS 在 FreeArt-21 上对轴的平均准确度约为 1 度,几何误差低于 1 cm,覆盖 21 个对象的平均值。
- 消融实验中,移除平滑性、初始正则化、抗噪性或混合渲染会显著降低关节/几何/外观指标。
- 在 Video2Articulation-S 上,FreeArtGS 在轴、状态和 Chamfer 距离等方面显著优于基线,同时保持竞争性的渲染质量。
- 真实世界实验显示对六个对象的关节类型/轴预测准确,几何/纹理重建鲁棒。
- 端到端优化中的混合渲染相较硬分配变体,PSNR 提升约 2 dB。
- 该方法在现有重建设置中保持竞争力,同时在提出的自由移动场景中表现出色。
阅读本论文
在 Nubint 阅读器中打开原文,边读边就任意段落提问。
在 Nubint 中阅读本论文本解读由 AI 根据论文原文生成,引用前请核对原文。