[Paper Review] Articulated Pose Estimation Using Hierarchical Exemplar-Based Models

Jiongxin Liu, Yinxiao Li+2 more|arXiv (Cornell University)|Dec 13, 2015

Human Pose and Action Recognition3 citations

TL;DR

This paper proposes a hierarchical exemplar-based model enhanced with deep convolutional neural networks (DCNNs) for articulated pose estimation, leveraging multi-level spatial constraints and part relations across granularities. By combining DCNNs for appearance modeling with hierarchical exemplars for spatial reasoning, the method achieves state-of-the-art performance on the Leeds Sports and CUB-200-2011 datasets, outperforming prior methods in part localization accuracy.

ABSTRACT

Exemplar-based models have achieved great success on localizing the parts of semi-rigid objects. However, their efficacy on highly articulated objects such as humans is yet to be explored. Inspired by hierarchical object representation and recent application of Deep Convolutional Neural Networks (DCNNs) on human pose estimation, we propose a novel formulation that incorporates both hierarchical exemplar-based models and DCNNs in the spatial terms. Specifically, we obtain more expressive spatial models by assuming independence between exemplars at different levels in the hierarchy; we also obtain stronger spatial constraints by inferring the spatial relations between parts at the same level. As our method strikes a good balance between expressiveness and strength of spatial models, it is both effective and generalizable, achieving state-of-the-art results on different benchmarks: Leeds Sports Dataset and CUB-200-2011.

Motivation & Objective

• To address the challenge of accurate articulated pose estimation in highly deformable objects like humans and birds.
• To improve the expressiveness of exemplar-based models, which traditionally underperform on highly articulated objects due to limited training data.
• To integrate deep learning-based appearance models (DCNNs) with hierarchical spatial constraints for stronger, more generalizable pose estimation.
• To develop an efficient inference framework that balances model expressiveness and spatial consistency across multiple levels of granularity.

Proposed method

• The method uses a hierarchical tree structure where atomic parts (e.g., joints) are at the finest level, and composite parts (e.g., limbs) are formed from groups of atomic parts at higher levels.
• Spatial relations between parts at the same level are modeled using DCNNs to predict relative offsets, while relations across levels follow parent-child hierarchies enforced via exemplar-based inference.
• Exemplars are used to encode plausible part configurations at each level, with independence assumed between exemplars at different levels to improve expressiveness.
• Spatial constraints are enforced by jointly optimizing part locations using a scoring function that combines appearance scores from DCNNs and spatial consistency from hierarchical exemplars.
• The approach employs bottom-up inference through the hierarchy, maintaining spatial coherence by propagating constraints from coarser to finer levels.
• The method uses grouped part optimization to jointly refine part locations, differing from prior approaches that optimize parts independently.

Experimental results

Research questions

• RQ1 Can exemplar-based models be made more expressive for highly articulated objects like humans by introducing hierarchical structure?
• RQ2 How can deep convolutional neural networks be effectively integrated into spatial modeling for articulated pose estimation?
• RQ3 Does combining multi-level spatial constraints with DCNN-based appearance models improve localization accuracy and generalization?
• RQ4 Can a hierarchical exemplar framework outperform non-hierarchical or tree-structured models in part localization under large deformations?

Key findings

• The proposed method achieves state-of-the-art performance on the Leeds Sports dataset, outperforming prior methods in Percentage of Detected Joints (PDJ) across most body parts.
• On the CUB-200-2011 bird dataset, the method achieves the highest PCP (Part Confidence Precision) scores, with Ours-full outperforming both DCNN-CoE and Part-pair baselines.
• The Ours-base variant, using only DCNN-based appearance models without spatial constraints, already outperforms previous methods like ( ? ) in most joints, demonstrating the strength of DCNNs for part detection.
• The full method (Ours-full) achieves significant gains over Ours-base by incorporating multi-level spatial modeling, especially in parts prone to deformation such as legs and wings.
• Qualitative results show robust localization across diverse poses, viewpoints, and deformations, with only a few failures due to extreme occlusion or noise.
• The method is efficient and compatible with off-the-shelf DCNN models, unlike more complex architectures required by competing methods.