[Paper Review] Movement science needs different pose tracking algorithms

Nidhi Seethapathi, Shaofei Wang+3 more|arXiv (Cornell University)|Jul 24, 2019

Human Pose and Action Recognition88 references66 citations

TL;DR

The paper argues that current pose tracking methods fail to prioritize movement-science-relevant quantities and outlines how to redesign algorithms to better estimate 3D kinematics, forces, and other movement-relevant variables for movement science.

ABSTRACT

Over the last decade, computer science has made progress towards extracting body pose from single camera photographs or videos. This promises to enable movement science to detect disease, quantify movement performance, and take the science out of the lab into the real world. However, current pose tracking algorithms fall short of the needs of movement science; the types of movement data that matter are poorly estimated. For instance, the metrics currently used for evaluating pose tracking algorithms use noisy hand-labeled ground truth data and do not prioritize precision of relevant variables like three-dimensional position, velocity, acceleration, and forces which are crucial for movement science. Here, we introduce the scientific disciplines that use movement data, the types of data they need, and discuss the changes needed to make pose tracking truly transformative for movement science.

Motivation & Objective

• Explain how movement science disciplines rely on movement data and why current pose tracking falls short.
• Identify the specific quantities movement science needs from pose tracking (3D position, velocity, acceleration, forces, absolute size).
• Propose concrete algorithmic and benchmarking changes to prioritize movement-relevant outputs.
• Highlight data and ground-truth considerations that would better support movement-science objectives.

Proposed method

• Survey diverse movement-science disciplines and their data needs.
• Critically analyze current pose tracking benchmarks and ground-truth practices for their lack of movement-relevant metrics.
• Propose algorithmic priors based on skeletal hierarchy, temporal structure, and camera motion to improve 3D kinematics.
• Recommend incorporating velocity, acceleration, and force estimation into objective functions and benchmarks.
• Suggest data augmentation and synthetic scenarios to handle contact and occlusion.
• Advise on achieving absolute size/mass estimates and fixed-frame references for movement analyses.

Experimental results

Research questions

• RQ1 What movement-relevant quantities (e.g., 3D kinematics, forces) should pose tracking algorithms prioritize for movement science?
• RQ2 How can pose tracking integrate temporal dynamics and skeletal constraints to improve accuracy over time?
• RQ3 What ground-truth data, benchmarks, and cross-population considerations are needed to avoid biases and support diverse movement tasks?
• RQ4 How can pose tracking provide absolute size, mass, and forces to enable mechanistic analyses in movement science?
• RQ5 What accommodations are needed for contact, occlusion, and camera motion to make in-the-wild data usable for movement science?

Key findings

• Current pose tracking emphasizes 2D pose and frame-wise accuracy, ignoring velocity, acceleration, and forces critical to movement science.
• Ground-truth benchmarks often rely on hand-labeled keypoints and contrived poses, not reflecting the movement quantities needed for science.
• Temporal information and dynamical structure are underutilized, leading to frame-to-frame errors that propagate to velocity/acceleration estimates.
• External forces, absolute mass, size, and inertia are not adequately estimated by existing methods, hindering biomechanical analyses.
• Accounting for camera motion and providing results in a fixed reference frame are essential for meaningful movement-science applications.
• Training data lacking demographic and task diversity can introduce biases in pose estimates across populations and activities.