[Paper Review] Kinematics-Guided Reinforcement Learning for Object-Aware 3D Ego-Pose Estimation

Zhengyi Luo, Ryo Hachiuma+3 more|arXiv (Cornell University)|Nov 10, 2020

Human Pose and Action Recognition25 references4 citations

TL;DR

This paper proposes a kinematics-guided reinforcement learning framework for 3D ego-pose estimation from egocentric videos, combining a kinematic pose estimator with a physics-based dynamics model in a residual action space to produce physically valid human-object interactions. The method achieves state-of-the-art performance with 31–48% lower error in root and joint position estimation compared to baselines, especially excelling in drift correction and generalization to in-the-wild settings.

ABSTRACT

We propose a method for incorporating object interaction and human body dynamics into the task of 3D ego-pose estimation using a head-mounted camera. We use a kinematics model of the human body to represent the entire range of human motion, and a dynamics model of the body to interact with objects inside a physics simulator. By bringing together object modeling, kinematics modeling, and dynamics modeling in a reinforcement learning (RL) framework, we enable object-aware 3D ego-pose estimation. We devise several representational innovations through the design of the state and action space to incorporate 3D scene context and improve pose estimation quality. We also construct a fine-tuning step to correct the drift and refine the estimated human-object interaction. This is the first work to estimate a physically valid 3D full-body interaction sequence with objects (e.g., chairs, boxes, obstacles) from egocentric videos. Experiments with both controlled and in-the-wild settings show that our method can successfully extract an object-conditioned 3D ego-pose sequence that is consistent with the laws of physics.

Motivation & Objective

• Address the challenge of estimating 3D full-body human motion and human-object interactions from a single first-person camera view, where the body is often occluded.
• Overcome the limitations of purely kinematic or dynamics-based approaches by combining accurate kinematic pose estimation with physics-consistent dynamics refinement.
• Incorporate 3D scene context—particularly object poses—into the reinforcement learning state representation to improve interaction prediction.
• Correct simulation drift in long-horizon interactions using a fine-tuning step aligned with monocular visual-inertial odometry (VIO) from real egocentric video.
• Demonstrate generalization to real-world, in-the-wild settings where ground-truth 3D poses are unavailable.

Proposed method

• Use a pre-trained kinematic pose estimator to provide a strong initial pose estimate, which is then used as a reference for the RL policy to predict residual joint angles instead of absolute joint targets.
• Design a novel action space in the RL policy that outputs a residual adjustment to the kinematic model’s output, enabling more stable and accurate policy learning.
• Integrate 6-DoF object pose as a state input to the RL agent, making the policy object-aware and contextually informed about scene interactions.
• Train a physics-based dynamics model in a simulator where the humanoid’s motion is controlled by joint torques, ensuring physically plausible interactions (e.g., pushing, sitting).
• Apply a fine-tuning stage using VIO-derived camera motion to correct global root trajectory drift in the simulation, aligning simulated motion with real-world camera movement.
• Design a multi-component reward function during fine-tuning that includes head position/orientation/velocity, kinematic regularization, and action smoothness to stabilize learning and improve realism.

Experimental results

Research questions

• RQ1 Can a hybrid kinematics-guided reinforcement learning framework improve the accuracy and physical plausibility of 3D ego-pose estimation from egocentric video compared to purely kinematic or dynamics-based methods?
• RQ2 How does incorporating 3D object pose information into the RL state space affect the quality and realism of human-object interaction estimation?
• RQ3 To what extent can a fine-tuning step using VIO data mitigate simulation drift and improve generalization to real-world egocentric videos?
• RQ4 What is the impact of different reward components (e.g., head tracking, regularization) on the final pose estimation performance and motion smoothness?
• RQ5 Can the proposed method generalize to in-the-wild settings with significant domain shift, where no ground-truth 3D poses are available?

Key findings

• The proposed residual action representation reduced root mean square error (RMSE) by 31% compared to baseline methods that directly predict PD targets.
• The ablation study showed that the fine-tuning step with full reward design (including head, kinematic, and action regularization) achieved the lowest error: 0.322 RMSE for root position, 0.788 for joint position, and 5.063 for velocity.
• In the in-the-wild evaluation, the method outperformed baselines by a large margin, especially in actions involving drift (e.g., pushing, avoiding obstacles), with a 34% reduction in root error compared to the next best method.
• The fine-tuned policy successfully corrected global trajectory drift, enabling reliable simulation of complex interactions like sitting on a chair or pushing a box.
• The method produced significantly smoother pose sequences (2.0x smoother) than baseline methods, as measured by acceleration and velocity metrics.
• Despite strong performance, failure cases occurred in in-the-wild data due to poor initial pose regressor output, leading to unnatural motions that the fine-tuning stage struggled to correct.