[Paper Review] Distribution-Aware Coordinate Representation for Human Pose Estimation

Feng Zhang, Xiatian Zhu+3 more|arXiv (Cornell University)|Oct 14, 2019

Human Pose and Action Recognition32 references43 citations

TL;DR

DARK introduces a distribution-aware coordinate representation for heatmap-based pose estimation, improving decoding and encoding to boost accuracy with plug-in compatibility across models and datasets (MPII, COCO).

ABSTRACT

While being the de facto standard coordinate representation in human pose estimation, heatmap is never systematically investigated in the literature, to our best knowledge. This work fills this gap by studying the coordinate representation with a particular focus on the heatmap. Interestingly, we found that the process of decoding the predicted heatmaps into the final joint coordinates in the original image space is surprisingly significant for human pose estimation performance, which nevertheless was not recognised before. In light of the discovered importance, we further probe the design limitations of the standard coordinate decoding method widely used by existing methods, and propose a more principled distribution-aware decoding method. Meanwhile, we improve the standard coordinate encoding process (i.e. transforming ground-truth coordinates to heatmaps) by generating accurate heatmap distributions for unbiased model training. Taking the two together, we formulate a novel Distribution-Aware coordinate Representation of Keypoint (DARK) method. Serving as a model-agnostic plug-in, DARK significantly improves the performance of a variety of state-of-the-art human pose estimation models. Extensive experiments show that DARK yields the best results on two common benchmarks, MPII and COCO, consistently validating the usefulness and effectiveness of our novel coordinate representation idea.

Motivation & Objective

• Highlight the impact of coordinate representation (encoding/decoding) on pose estimation performance.
• Propose a principled, distribution-aware decoding method using a Gaussian assumption and Taylor expansion.
• Address quantisation/heatmap distribution issues during encoding to provide unbiased supervision.
• Demonstrate DARK as a model-agnostic plug-in that improves state-of-the-art models on COCO and MPII.

Proposed method

• Identify the significance of heatmap decoding and propose distribution-aware decoding based on a 2D Gaussian model for sub-pixel localization.
• Apply Taylor expansion around the heatmap maximum to estimate the true joint center (μ) with first and second derivatives.
• Introduce heatmap distribution modulation to better resemble training Gaussian distributions via Gaussian kernel smoothing.
• Provide unbiased heatmap encoding by centering Gaussians at sub-pixel ground-truth coordinates to remove quantisation bias.
• Demonstrate DARK as a plug-in compatible with existing models (e.g., HRNet, SimpleBaseline, Hourglass) without architectural changes.

Experimental results

Research questions

• RQ1 How does coordinate decoding (and its conventional shifts) affect pose estimation accuracy across models?
• RQ2 Can a distribution-aware decoding method improve sub-pixel localization beyond standard shifting?
• RQ3 Does Gaussian-based heatmap distribution modulation improve decoding under real predictions?
• RQ4 Does unbiased sub-pixel heatmap encoding provide measurable supervision benefits?
• RQ5 Is DARK generalizable as a model-agnostic plug-in across different pose estimation architectures?

Key findings

• Standard coordinate decoding with shifting yields up to 5.7% AP improvement over no-shift decoding for HRNet-W32 at 128x96, while DARK provides additional gains.
• Applying distribution modulation (DM) to heatmaps improves AP from 68.1 to 68.4 on COCO val with HRNet-W32 at 128x96.
• Unbiased heatmap encoding with DARK decodings yields AP of 70.7 (vs. 66.9 biased) on COCO val for HRNet-W32 at 128x96.
• DARK at 128x96 with HRNet-W32 achieves AP 70.7 and related metrics, and at higher input size (256x192, 384x288) further boosts performance over baselines (e.g., 74.4/75.8 vs. 74.4/73.7 depending on setup).
• On COCO test-dev, DARK with HRNet-W48 at 384x288 reaches AP 76.2, surpassing the best competitor by 0.7 AP points (76.2 vs. 75.5).
• MPII results show DARK improves mean PCKh@0.5 to 90.6 and PCKh@0.1 to 42.0, outperforming HRNet-W32 baselines.