[论文解读] AlphaPose: Whole-Body Regional Multi-Person Pose Estimation and Tracking in Real-Time
AlphaPose 提供一个实时系统用于多人场景的全身(面部、身体、手、脚)姿态估计与跟踪,提出对称积分关键点回归、基于姿态的非极大抑制、姿态感知身份嵌入,以及处理不完美检测与来自多领域数据的训练策略。
Accurate whole-body multi-person pose estimation and tracking is an important yet challenging topic in computer vision. To capture the subtle actions of humans for complex behavior analysis, whole-body pose estimation including the face, body, hand and foot is essential over conventional body-only pose estimation. In this paper, we present AlphaPose, a system that can perform accurate whole-body pose estimation and tracking jointly while running in realtime. To this end, we propose several new techniques: Symmetric Integral Keypoint Regression (SIKR) for fast and fine localization, Parametric Pose Non-Maximum-Suppression (P-NMS) for eliminating redundant human detections and Pose Aware Identity Embedding for jointly pose estimation and tracking. During training, we resort to Part-Guided Proposal Generator (PGPG) and multi-domain knowledge distillation to further improve the accuracy. Our method is able to localize whole-body keypoints accurately and tracks humans simultaneously given inaccurate bounding boxes and redundant detections. We show a significant improvement over current state-of-the-art methods in both speed and accuracy on COCO-wholebody, COCO, PoseTrack, and our proposed Halpe-FullBody pose estimation dataset. Our model, source codes and dataset are made publicly available at https://github.com/MVIG-SJTU/AlphaPose.
研究动机与目标
- 实现包括面部、手和脚在内的复杂行为的高精度、实时全身姿态估计。
- 构建一个自上而下框架,在不完美检测和冗余提议下保持鲁棒。
- 引入新技术以提升关键点定位、检测融合和跨帧身份跟踪。
- 提供训练策略和新数据集,推进野外场景的全身姿态估计。
提出的方法
- Symmetric Integral Keypoint Regression (SIKR) 以实现快速且准确的关键点定位,降低量化误差。
- Parametric Pose Non-Maximum-Suppression (P-NMS) 使用学习到的姿态距离度量去除冗余检测。
- Pose-Aware Identity Embedding with Pose-Guided Attention 使联合姿态估计与跟踪成为可能。
- Part-Guided Proposal Generator (PGPG) 通过模拟身体部位的检测输出分布来扩充训练数据。
- Multi-Domain Knowledge Distillation 借助如 300Wface、FreiHand、InterHand 等数据集实现更好的泛化。
- 两步热图归一化策略将置信度估计与积分回归解耦,降低关键点打分对尺度的依赖。
实验结果
研究问题
- RQ1如何在自上而下框架内实现对多人的实时全身姿态估计?
- RQ2我们能否设计能够有效处理不同尺度和不完美检测的关键点回归与 NMS 方法?
- RQ3哪些训练策略可以提升在野外数据和多领域联合定位的泛化能力?
- RQ4如何利用姿态信息改进跨帧的重新识别与跟踪?
主要发现
- 提出的方法 SIKR 解决量化与尺度变化问题,实现身体、面部、手部的关节点准确定位。
- 使用数据驱动姿态距离的 P-NMS 提高冗余移除和最终姿态选择。
- 姿态引导的注意力和 MSIM 使在线实时跟踪成为可能,整合姿态、框和 re-id 特征。
- PGPG 和多领域蒸馏提高对检测噪声和跨领域数据的鲁棒性,在 COCO-wholebody、COCO、PoseTrack、Halpe-FullBody 数据集上提升性能。
- AlphaPose 在多个基准上相对于最先进方法在速度和准确度上均有改进,且公开代码和数据集。
阅读本论文
在 Nubint 阅读器中打开原文,边读边就任意段落提问。
在 Nubint 中阅读本论文本解读由 AI 根据论文原文生成,引用前请核对原文。