[论文解读] Landmark Weighting for 3DMM Shape Fitting

Yu Yanga, Xiaojun Wu+1 more|arXiv (Cornell University)|Aug 16, 2018

Face recognition and analysis参考文献 19被引用 6

一句话总结

该论文提出了一种新颖的3DMM形状拟合方法，通过基于个体估计误差自适应加权2D-3D关键点对应关系，提升了重建精度。与传统方法中对所有关键点一视同仁的做法不同，该方法为更可靠的的关键点分配更高的权重，从而降低拟合误差，并通过误差感知优化增强3D人脸模型的真实性。

ABSTRACT

Human face is a 3D object with shape and surface texture. 3D Morphable Model (3DMM) is a powerful tool for reconstructing the 3D face from a single 2D face image. In the shape fitting process, 3DMM estimates the correspondence between 2D and 3D landmarks. Most traditional 3DMM fitting methods fail to reconstruct an accurate model because face shape fitting is a difficult non-linear optimization problem. In this paper we show that landmark weighting is instrumental to improve the accuracy of shape reconstruction and propose a novel 3D Morphable Model Fitting method. Different from previous works that treat all landmarks equally, we take into consideration the estimated errors for each pair of 2D and 3D corresponding landmarks. The landmark points are weighted in the optimization cost function based on these errors. Obviously, these landmarks have different semantics because they locate on different facial components. In the context of the solution of fitting is approximated, there are deviations in landmarks matching. However, these landmarks with different semantics have different effects on reconstructing 3D faces. Thus, it is necessary to consider each landmark individually. To our knowledge, we are the first to analyze each feature point for 3D face reconstruction by 3DMM. The weight is adaptive with the estimation residuals of landmarks. Experimental results show that the proposed method significantly reduces the reconstruction error and improves the authenticity of the 3D model expression.

研究动机与目标

• 解决由于3DMM拟合中非线性优化导致的3D人脸重建不准确的问题。
• 认识到传统3DMM方法对所有关键点同等对待，忽略了语义和可靠性差异。
• 通过建模关键点特定的估计误差并相应调整权重，提升形状拟合精度。
• 研究关键点语义和匹配偏差对3D人脸重建质量的影响。
• 在优化目标函数中提出一种新颖的自适应加权方案，以提升重建保真度。

提出的方法

• 在3DMM形状拟合的优化过程中引入关键点加权机制。
• 为每个2D-3D关键点对应关系估计残差误差，以评估其可靠性。
• 根据其个体估计残差，在代价函数中为关键点分配自适应权重。
• 修改标准3DMM优化目标，优先考虑匹配不确定性较低的关键点。
• 采用非线性优化框架，使关键点权重在拟合过程中动态调整。
• 利用面部关键点之间的语义差异（例如鼻尖与眼角）来指导加权策略。

实验结果

研究问题

• RQ1 对关键点不加区分的处理方式如何影响3DMM拟合中3D人脸重建的准确性？
• RQ2 基于估计误差的自适应关键点加权能否提升3DMM形状拟合性能？
• RQ3 关键点语义和匹配偏差对重建质量有何影响？
• RQ4 与均匀加权相比，误差感知加权在重建误差和真实感方面表现如何？
• RQ5 所提方法能否在降低拟合误差的同时，保持3D人脸模型的自然面部表情？

主要发现

• 与基线3DMM拟合方法相比，所提出的加权方法显著降低了3D人脸重建误差。
• 基于估计残差的自适应加权可生成更精确、更逼真的3D人脸模型。
• 匹配不确定性较高的关键点会被自然地降低权重，从而减少其对优化的影响。
• 该方法通过更好地保留面部几何结构和表情细节，提升了模型的真实性。
• 实验结果表明，该方法在多个基准数据集上均表现出一致的性能提升。
• 该方法是首个明确基于语义和误差标准分析并加权个体关键点的3DMM拟合方法。