[Paper Review] A Spatial-temporal 3D Human Pose Reconstruction Framework

Xuan Nguyen, Thi Duyen Ngo+1 more|arXiv (Cornell University)|Jan 8, 2019

Human Pose and Action Recognition11 references4 citations

TL;DR

This paper proposes a spatial-temporal 3D human pose reconstruction framework that leverages both intra-frame and inter-frame correlations in 2D pose sequences using Orthogonal Matching Pursuit (OMP), pre-trained pose-angle limits, and temporal modeling. It achieves 10% lower Euclidean reconstruction error and improved robustness to Gaussian noise on CMU Mocap and Vietnamese traditional dance datasets, with smoother, more natural 3D pose sequences compared to state-of-the-art methods.

ABSTRACT

3D human pose reconstruction from single-view camera is a difficult and challenging topic. Many approaches have been proposed, but almost focusing on frame-by-frame independently while inter-frames are highly correlated in a pose sequence. In contrast, we introduce a novel spatial-temporal 3D reconstruction framework that leverages both intra and inter frame relationships in consecutive 2D pose sequences. Orthogonal Matching Pursuit (OMP) algorithm, pre-trained Pose-angle Limits and Temporal Models have been implemented. We quantitatively compare our framework versus recent works on CMU motion capture dataset and Vietnamese traditional dance sequences. Our method outperforms others with 10 percent lower of Euclidean reconstruction error and robustness against Gaussian noise. Additionally, it is also important to mention that our reconstructed 3D pose sequences are smoother and more natural than others.

Motivation & Objective

• To address the challenge of 3D human pose reconstruction from single-view 2D poses, which often treats frames in isolation despite strong temporal correlations.
• To improve reconstruction accuracy and temporal smoothness by modeling both spatial relationships within frames and temporal dependencies across consecutive frames.
• To enhance robustness against noise and produce more natural, realistic 3D pose sequences.
• To outperform existing frame-by-frame methods by integrating spatial and temporal priors through a unified framework.

Proposed method

• Utilizes Orthogonal Matching Pursuit (OMP) to reconstruct 3D poses by selecting optimal basis vectors from a learned dictionary of pose configurations.
• Incorporates pre-trained pose-angle limits as spatial constraints to enforce anatomically plausible joint configurations within each frame.
• Applies temporal models to exploit sequential dependencies between consecutive 2D pose sequences, improving temporal coherence.
• Combines spatial and temporal priors into a unified optimization framework that jointly refines 3D pose estimates across time.
• Employs a multi-stage refinement process that alternates between spatial consistency enforcement and temporal smoothing.
• Leverages the CMU Mocap dataset and custom Vietnamese traditional dance sequences for training and evaluation.

Experimental results

Research questions

• RQ1 Can integrating temporal dependencies between consecutive 2D pose frames improve 3D reconstruction accuracy compared to frame-independent methods?
• RQ2 How does the incorporation of anatomically plausible pose-angle limits affect reconstruction quality and stability?
• RQ3 To what extent does the proposed framework reduce Euclidean reconstruction error on benchmark datasets?
• RQ4 How does the method perform under noisy conditions, particularly with Gaussian noise?
• RQ5 Does the framework produce smoother and more natural 3D pose sequences than existing state-of-the-art approaches?

Key findings

• The proposed framework achieves a 10% reduction in mean Euclidean reconstruction error compared to recent state-of-the-art methods on the CMU Mocap dataset.
• The method demonstrates superior robustness to Gaussian noise, maintaining lower error rates under noisy 2D pose inputs.
• Reconstructed 3D pose sequences are significantly smoother and more temporally coherent than those from competing methods.
• The integration of pose-angle limits and temporal modeling leads to more anatomically plausible 3D poses.
• The framework generalizes well to diverse motion styles, including complex sequences from Vietnamese traditional dance.
• Quantitative evaluation confirms consistent performance gains across both benchmark and custom datasets.