[Paper Review] CrowdPose: Efficient Crowded Scenes Pose Estimation and A New Benchmark

Jiefeng Li, Can Wang+4 more|arXiv (Cornell University)|Dec 2, 2018

Human Pose and Action RecognitionComputer Science28 references41 citations

TL;DR

The paper introduces joint-candidate SPPE and a global maximum joints association to improve multi-person pose estimation in crowded scenes, and presents the CrowdPose dataset; it achieves a 5.2 mAP gain on CrowdPose and generalizes to MSCOCO.

ABSTRACT

Multi-person pose estimation is fundamental to many computer vision tasks and has made significant progress in recent years. However, few previous methods explored the problem of pose estimation in crowded scenes while it remains challenging and inevitable in many scenarios. Moreover, current benchmarks cannot provide an appropriate evaluation for such cases. In this paper, we propose a novel and efficient method to tackle the problem of pose estimation in the crowd and a new dataset to better evaluate algorithms. Our model consists of two key components: joint-candidate single person pose estimation (SPPE) and global maximum joints association. With multi-peak prediction for each joint and global association using graph model, our method is robust to inevitable interference in crowded scenes and very efficient in inference. The proposed method surpasses the state-of-the-art methods on CrowdPose dataset by 5.2 mAP and results on MSCOCO dataset demonstrate the generalization ability of our method. Source code and dataset will be made publicly available.

Motivation & Objective

• Motivate improved pose estimation in crowded scenes where occlusion and overlap degrade performance.
• Develop a fast, robust method that leverages global information to resolve ambiguous joints.
• Create a dataset with uniform crowding to evaluate performance across uncrowded and crowded scenes.
• Demonstrate generalization of the approach on MSCOCO and provide publicly available code and data.

Proposed method

• Use a two-stage top-down pipeline with human proposals and single-person pose estimation.
• Introduce joint-candidate SPPE that predicts multiple joint candidates per joint, incorporating target and interference joints.
• Construct a person-joint graph linking human proposals and joint candidates.
• Apply a global maximum joints association using a graph matching formulation decomposed into K bipartite subproblems solved by Kuhn–Munkres algorithm.
• Ensure computational efficiency by proving the approach has hardness comparable to greedy NMS and leveraging sparsity.

Experimental results

Research questions

• RQ1 How can pose estimation in crowded scenes be robust to interference from nearby persons?
• RQ2 Can a joint-candidate, multi-peak approach improve recall of true joints without sacrificing precision?
• RQ3 Does a global, graph-based association improve correct joint-to-person assignment over local or greedy methods?
• RQ4 How well does the proposed CrowdPose dataset support evaluation across uncrowded and crowded scenes and promote generalization?

Key findings

• The proposed method surpasses state-of-the-art on CrowdPose by 5.2 mAP with the same backbone.
• On MSCOCO, replacing SPPE and post-processing with the proposed modules yields +0.8 mAP.
• The method achieves 10.1 FPS on the CrowdPose test set, balancing accuracy and speed in crowded scenarios.
• CrowdPose dataset displays a uniform Crowd Index distribution and higher average bounding box IoU (0.27) than MSCOCO, MPII, and AI Challenger, supporting diverse evaluation.
• The joint-candidate loss significantly improves mAP (66.0%) vs using standard loss (61.7%).