[Paper Review] Simple Baselines for Human Pose Estimation and Tracking

Bin Xiao, Haiping Wu+1 more|arXiv (Cornell University)|Apr 17, 2018

Human Pose and Action RecognitionComputer Science23 references118 citations

TL;DR

The paper presents simple, strong baselines for single-frame human pose estimation and multi-person pose tracking in video, achieving state-of-the-art results with a lightweight deconvolution head and flow-based tracking.

ABSTRACT

There has been significant progress on pose estimation and increasing interests on pose tracking in recent years. At the same time, the overall algorithm and system complexity increases as well, making the algorithm analysis and comparison more difficult. This work provides simple and effective baseline methods. They are helpful for inspiring and evaluating new ideas for the field. State-of-the-art results are achieved on challenging benchmarks. The code will be available at https://github.com/leoxiaobin/pose.pytorch.

Motivation & Objective

• Investigate whether simple head-designs with deconvolution can yield competitive heatmaps for multi-person pose estimation.
• Develop a practical flow-based tracking pipeline that propagates joints via optical flow and uses flow-based similarity for data association.
• Evaluate on COCO for single-frame pose estimation and on PoseTrack for video-based pose estimation and tracking.
• Provide ablations to understand the impact of heatmap resolution, backbone depth, input size, and flow-based components.

Proposed method

• Add a small stack of deconvolution layers on top of a ResNet backbone to convert deep features into heatmaps for keypoint prediction.
• Train with MSE loss between predicted and Gaussian-smoothed ground-truth heatmaps.
• For tracking, propagate joints from previous frames using optical flow to generate candidate boxes and compute a flow-based pose similarity (OKS between propagated joints and current joints).
• Unify detector boxes and propagated boxes via NMS, estimate poses per unified box, and perform greedy ID assignment using a flow-based similarity matrix.
• Extend the tracking with a multi-frame flow-based similarity to link across occlusions and brief disappearances.
• Provide an online inference algorithm that maintains a short history queue of past frames to compute M_sim and assign IDs.

Experimental results

Research questions

• RQ1 How far can a simple deconvolution head on top of a ResNet backbone push pose estimation accuracy on COCO without complex multi-stage architectures?
• RQ2 Can optical-flow-based joint propagation and flow-based pose similarity yield competitive pose tracking in videos compared to state-of-the-art methods that rely on more complex pipelines?
• RQ3 What is the impact of backbone depth, input resolution, and deconvolution kernel choices on pose estimation performance?
• RQ4 Does integrating joint propagation with detector boxes improve detection and tracking robustness in challenging video scenarios?
• RQ5 How does flow-based multi-frame similarity affect Tracking-By-Detection performance on PoseTrack?

Key findings

• On COCO test-dev, the authors’ single-model baseline achieves 73.7 AP, matching or surpassing prior single-model and ensemble results.
• The flow-based tracking extension yields a new state-of-the-art on PoseTrack with MOTA 57.8 (ResNet-152, 384×288) and 74.6 mAP for multi-person pose tracking on Leaderboard entries.
• Ablations show heatmap resolution and larger input size improve AP; deeper backbones improve performance (e.g., ResNet-152).
• Joint propagation from optical flow significantly improves both mAP and MOTA across detectors (R-FCN and FPN-DCN) and backbones, especially for weaker detectors.
• Flow-based pose similarity (including multi-frame) outperforms bounding-box or pose-based similarity, particularly under fast motion or occlusion.
• The proposed baseline outperforms several contemporary methods on PoseTrack validation and test sets, demonstrating strong practical performance with simpler design.