[Paper Review] Bottom-Up Human Pose Estimation Via Disentangled Keypoint Regression

Zigang Geng, Ke Sun+3 more|arXiv (Cornell University)|Apr 6, 2021

Human Pose and Action Recognition77 references34 citations

TL;DR

Introduces DEKR, a bottom-up pose estimation method that uses multi-branch adaptive convolutions to disentangle keypoint representations and directly regress keypoint positions, achieving state-of-the-art results on COCO and CrowdPose.

ABSTRACT

In this paper, we are interested in the bottom-up paradigm of estimating human poses from an image. We study the dense keypoint regression framework that is previously inferior to the keypoint detection and grouping framework. Our motivation is that regressing keypoint positions accurately needs to learn representations that focus on the keypoint regions. We present a simple yet effective approach, named disentangled keypoint regression (DEKR). We adopt adaptive convolutions through pixel-wise spatial transformer to activate the pixels in the keypoint regions and accordingly learn representations from them. We use a multi-branch structure for separate regression: each branch learns a representation with dedicated adaptive convolutions and regresses one keypoint. The resulting disentangled representations are able to attend to the keypoint regions, respectively, and thus the keypoint regression is spatially more accurate. We empirically show that the proposed direct regression method outperforms keypoint detection and grouping methods and achieves superior bottom-up pose estimation results on two benchmark datasets, COCO and CrowdPose. The code and models are available at https://github.com/HRNet/DEKR.

Motivation & Objective

• Motivate improving bottom-up pose estimation by focusing representations on keypoint regions rather than relying on heatmap-based detection and grouping.
• Propose a disentangled regression framework (DEKR) with adaptive convolutions and a multi-branch architecture to regress each keypoint separately.
• Demonstrate that direct keypoint regression with DEKR outperforms traditional keypoint detection and grouping on COCO and CrowdPose.

Proposed method

• Use pixel-wise spatial transformer inspired adaptive convolutions to activate keypoint-region pixels.
• Employ a multi-branch structure where each branch learns representations dedicated to one keypoint and regresses its 2D offset.
• Train with a joint regression loss and heatmap losses for keypoints and centers, plus a weighted OKS-based evaluation.
• Perform center-based and pose-based non-maximum suppression during inference, with a pose scoring network to rank candidates.

Experimental results

Research questions

• RQ1 Can disentangled, per-keypoint representations improve direct keypoint regression quality in bottom-up pose estimation?
• RQ2 Does combining adaptive activations with a separate-regression branches yield higher localization accuracy than single-branch regression or grouping approaches?
• RQ3 What are the performance gains of DEKR on standard benchmarks COCO and CrowdPose compared to state-of-the-art bottom-up methods?

Key findings

• DEKR achieves state-of-the-art bottom-up pose estimation results on COCO and CrowdPose.
• Single-branch regression is outperformed by a multi-branch, disentangled approach that focuses on individual keypoint regions.
• Adaptive activation and separate regression together yield significant AP gains (e.g., 68.0 AP with HRNet-W32; 71.0 AP with HRNet-W48 on COCO val).
• On COCO val, DEKR with HRNet-W32 achieves 68.0 AP; with HRNet-W48 achieves 71.0 AP; multi-scale testing yields 71.0 AP (W32) and 72.8 AP (W48).
• On COCO test-dev, DEKR with HRNet-W32 achieves 67.3 AP; with HRNet-W48 achieves 70.0 AP, with multi-scale up to 69.8 and 71.0 respectively.
• On CrowdPose, DEKR achieves 65.5 AP (val, D-32 ss) and 67.0 AP (val, D-48 ss); multi-scale improves to 67.5 and 68.3 respectively.
• Ablations show adaptive activation contributes about 3.5 AP and separate regression adds about 2.6 AP on COCO val with HRNet-W32.
• Compared to grouping or post-hoc matching strategies (e.g., CenterNet’s absorbing scheme), DEKR provides consistent single-scale gains without needing heatmap-based post-processing to match keypoints.