[Paper Review] ArtTrack: Articulated Multi-person Tracking in the Wild

Eldar Insafutdinov, Mykhaylo Andriluka+5 more|arXiv (Cornell University)|Dec 5, 2016

Human Pose and Action Recognition8 references19 citations

TL;DR

ArtTrack proposes a fast, bottom-up approach for articulated multi-person tracking in unconstrained videos by combining sparse body-part graphs with a feed-forward convolutional network for joint-to-person association. It achieves state-of-the-art accuracy on MPII Video Pose and We Are Family datasets while being 24× faster than prior methods by leveraging temporal reasoning and efficient spatio-temporal grouping of proposals.

ABSTRACT

In this paper we propose an approach for articulated tracking of multiple people in unconstrained videos. Our starting point is a model that resembles existing architectures for single-frame pose estimation but is substantially faster. We achieve this in two ways: (1) by simplifying and sparsifying the body-part relationship graph and leveraging recent methods for faster inference, and (2) by offloading a substantial share of computation onto a feed-forward convolutional architecture that is able to detect and associate body joints of the same person even in clutter. We use this model to generate proposals for body joint locations and formulate articulated tracking as spatio-temporal grouping of such proposals. This allows to jointly solve the association problem for all people in the scene by propagating evidence from strong detections through time and enforcing constraints that each proposal can be assigned to one person only. We report results on a public MPII Human Pose benchmark and on a new MPII Video Pose dataset of image sequences with multiple people. We demonstrate that our model achieves state-of-the-art results while using only a fraction of time and is able to leverage temporal information to improve state-of-the-art for crowded scenes.

Motivation & Objective

• Address articulated multi-person tracking in unconstrained videos with occlusions, fast motion, and variable appearance.
• Overcome limitations of end-to-end models in crowded scenes with unknown numbers of people.
• Improve tracking accuracy by jointly reasoning about body part associations across time and space.
• Develop an efficient inference framework that maintains high accuracy while reducing computational cost.
• Create a new benchmark dataset, MPII Video Pose, for evaluating articulated tracking in realistic, crowded scenes.

Proposed method

• Use a sparse body-part relationship graph with only essential spatial connections to reduce computation.
• Train a feed-forward convolutional network to predict joint-to-person associations, offloading most reasoning from iterative optimization.
• Generate body-part proposals per frame using a top-down detection head and combine them via spatio-temporal graph partitioning.
• Formulate tracking as a joint assignment problem enforcing one-person-per-proposal constraints and inter-person exclusion.
• Incorporate temporal features such as detection distance, deep feature matching, and SIFT-based similarity to improve tracking consistency across frames.
• Apply local combinatorial optimization for efficient inference, enabling real-time performance.

Experimental results

Research questions

• RQ1 Can a sparse, fast inference model outperform fully connected models in articulated multi-person tracking?
• RQ2 How effective is feed-forward joint-to-person association in handling occlusions and complex scenes?
• RQ3 To what extent can temporal modeling improve tracking accuracy in crowded, unconstrained videos?
• RQ4 Can a bottom-up approach generalize to scenes with an unknown number of people and strong occlusions?
• RQ5 How do different temporal features (geometry, appearance, SIFT) contribute to tracking robustness?

Key findings

• ArtTrack achieves 73.1% AP on the new MPII Video Pose dataset, a 1.5% improvement over single-frame baseline using temporal features.
• The TD/BU model outperforms DeeperCut by 5.7% AP on the We Are Family dataset, especially on challenging parts like wrists and elbows.
• The proposed method is 24× faster than state-of-the-art methods while maintaining comparable or better accuracy.
• Temporal features such as deepmatch and SIFT-distance significantly improve performance, especially under fast motion and background changes.
• The model successfully reconstructs partially occluded limbs (e.g., legs in the back) through explicit top-down reasoning and temporal propagation.
• The sparse graph model performs competitively with fully connected models but with drastically reduced inference cost.