Skip to main content
QUICK REVIEW

[论文解读] AlphaPose: Whole-Body Regional Multi-Person Pose Estimation and Tracking in Real-Time

Haoshu Fang, Jiefeng Li|arXiv (Cornell University)|Nov 7, 2022
Human Pose and Action RecognitionComputer Science被引用 37
一句话总结

AlphaPose 提供一个实时系统用于多人场景的全身(面部、身体、手、脚)姿态估计与跟踪,提出对称积分关键点回归、基于姿态的非极大抑制、姿态感知身份嵌入,以及处理不完美检测与来自多领域数据的训练策略。

ABSTRACT

Accurate whole-body multi-person pose estimation and tracking is an important yet challenging topic in computer vision. To capture the subtle actions of humans for complex behavior analysis, whole-body pose estimation including the face, body, hand and foot is essential over conventional body-only pose estimation. In this paper, we present AlphaPose, a system that can perform accurate whole-body pose estimation and tracking jointly while running in realtime. To this end, we propose several new techniques: Symmetric Integral Keypoint Regression (SIKR) for fast and fine localization, Parametric Pose Non-Maximum-Suppression (P-NMS) for eliminating redundant human detections and Pose Aware Identity Embedding for jointly pose estimation and tracking. During training, we resort to Part-Guided Proposal Generator (PGPG) and multi-domain knowledge distillation to further improve the accuracy. Our method is able to localize whole-body keypoints accurately and tracks humans simultaneously given inaccurate bounding boxes and redundant detections. We show a significant improvement over current state-of-the-art methods in both speed and accuracy on COCO-wholebody, COCO, PoseTrack, and our proposed Halpe-FullBody pose estimation dataset. Our model, source codes and dataset are made publicly available at https://github.com/MVIG-SJTU/AlphaPose.

研究动机与目标

  • 实现包括面部、手和脚在内的复杂行为的高精度、实时全身姿态估计。
  • 构建一个自上而下框架,在不完美检测和冗余提议下保持鲁棒。
  • 引入新技术以提升关键点定位、检测融合和跨帧身份跟踪。
  • 提供训练策略和新数据集,推进野外场景的全身姿态估计。

提出的方法

  • Symmetric Integral Keypoint Regression (SIKR) 以实现快速且准确的关键点定位,降低量化误差。
  • Parametric Pose Non-Maximum-Suppression (P-NMS) 使用学习到的姿态距离度量去除冗余检测。
  • Pose-Aware Identity Embedding with Pose-Guided Attention 使联合姿态估计与跟踪成为可能。
  • Part-Guided Proposal Generator (PGPG) 通过模拟身体部位的检测输出分布来扩充训练数据。
  • Multi-Domain Knowledge Distillation 借助如 300Wface、FreiHand、InterHand 等数据集实现更好的泛化。
  • 两步热图归一化策略将置信度估计与积分回归解耦,降低关键点打分对尺度的依赖。

实验结果

研究问题

  • RQ1如何在自上而下框架内实现对多人的实时全身姿态估计?
  • RQ2我们能否设计能够有效处理不同尺度和不完美检测的关键点回归与 NMS 方法?
  • RQ3哪些训练策略可以提升在野外数据和多领域联合定位的泛化能力?
  • RQ4如何利用姿态信息改进跨帧的重新识别与跟踪?

主要发现

  • 提出的方法 SIKR 解决量化与尺度变化问题,实现身体、面部、手部的关节点准确定位。
  • 使用数据驱动姿态距离的 P-NMS 提高冗余移除和最终姿态选择。
  • 姿态引导的注意力和 MSIM 使在线实时跟踪成为可能,整合姿态、框和 re-id 特征。
  • PGPG 和多领域蒸馏提高对检测噪声和跨领域数据的鲁棒性,在 COCO-wholebody、COCO、PoseTrack、Halpe-FullBody 数据集上提升性能。
  • AlphaPose 在多个基准上相对于最先进方法在速度和准确度上均有改进,且公开代码和数据集。

阅读本论文

在 Nubint 阅读器中打开原文,边读边就任意段落提问。

在 Nubint 中阅读本论文

本解读由 AI 根据论文原文生成,引用前请核对原文。