[论文解读] AMIL: Adversarial Multi Instance Learning for Human Pose Estimation

Pourya Shamsolmoali, Masoumeh Zareapoor+2 more|arXiv (Cornell University)|Mar 18, 2020

Human Pose and Action Recognition参考文献 65被引用 5

一句话总结

本文提出 AMIL，一种新颖的对抗性多实例学习框架，通过生成对抗网络（GAN）整合人体结构先验，实现对人体姿态估计的改进。该框架采用两个残差 MIL 模型——一个生成器和一个判别器。判别器用于区分真实与生成的热力图，对抗性训练引导生成器生成更具真实感、结构上更合理的姿态，显著提升了基准数据集上的准确率。

ABSTRACT

Human pose estimation has an important impact on a wide range of applications from human-computer interface to surveillance and content-based video retrieval. For human pose estimation, joint obstructions and overlapping upon human bodies result in departed pose estimation. To address these problems, by integrating priors of the structure of human bodies, we present a novel structure-aware network to discreetly consider such priors during the training of the network. Typically, learning such constraints is a challenging task. Instead, we propose generative adversarial networks as our learning model in which we design two residual multiple instance learning (MIL) models with the identical architecture, one is used as the generator and the other one is used as the discriminator. The discriminator task is to distinguish the actual poses from the fake ones. If the pose generator generates the results that the discriminator is not able to distinguish from the real ones, the model has successfully learnt the priors. In the proposed model, the discriminator differentiates the ground-truth heatmaps from the generated ones, and later the adversarial loss back-propagates to the generator. Such procedure assists the generator to learn reasonable body configurations and is proved to be advantageous to improve the pose estimation accuracy. Meanwhile, we propose a novel function for MIL. It is an adjustable structure for both instance selection and modeling to appropriately pass the information between instances in a single bag. In the proposed residual MIL neural network, the pooling action adequately updates the instance contribution to its bag. The proposed adversarial residual multi-instance neural network that is based on pooling has been validated on two datasets for the human pose estimation task and successfully outperforms the other state-of-arts models.

研究动机与目标

• 解决在人群密集或复杂场景中因关节遮挡和身体部位重叠导致的姿态估计不准确问题。
• 将人体结构配置的先验知识整合到深度学习模型中，以提升姿态估计的鲁棒性。
• 开发一种新颖的残差 MIL 架构，有效建模图像（袋）内实例（关键点）之间的关系，以实现更优的特征传播。
• 利用对抗性训练引导生成器生成更具真实感和解剖学合理性的热力图。
• 在标准基准数据集上，超越当前最先进方法，提升人体姿态估计的泛化能力和准确率。

提出的方法

• 在 GAN 框架中提出一种共享架构的残差多实例学习（MIL）网络，用于生成器和判别器。
• 使用判别器区分真实真实热力图与生成器生成的虚假热力图。
• 通过对抗性训练使生成器学习生成与真实热力图难以区分的输出，从而编码结构先验。
• 在 MIL 层中设计一种新型池化机制，自适应地更新实例对袋的贡献，提升信息流动。
• 将对抗性损失与标准姿态估计损失联合优化，兼顾真实感与准确率。
• 端到端地将模型应用于图像输入，预测人体 14 个关键点位置的热力图。

实验结果

研究问题

• RQ1 通过判别器进行对抗性训练，能否提升预测人体姿态热力图的真实感与结构一致性？
• RQ2 所提出的残差 MIL 层在单个图像（袋）内建模实例间关系方面是否有效？
• RQ3 通过对抗性学习引入结构先验，是否能在遮挡和重叠情况下带来可测量的姿态估计准确率提升？
• RQ4 所提出的 AMIL 模型在标准人体姿态估计基准上与最先进方法相比表现如何？
• RQ5 所提出的 MIL 池化机制是否能通过自适应加权关键点实例来增强特征表示？

主要发现

• 所提出的 AMIL 模型在两个标准人体姿态估计数据集上达到最先进性能，优于现有方法。
• 对抗性训练显著提升了预测关键点热力图的结构合理性，尤其在遮挡或重叠场景下表现更优。
• 结合自适应池化的残差 MIL 层通过动态调整袋内实例的贡献，增强了特征表示。
• 判别器能够有效区分真实与生成热力图，证实生成器学习到了有意义的结构先验。
• 该模型对关节遮挡和身体部位重叠具有鲁棒性，实现了更准确、更一致的姿态预测。
• 定量结果表明，与基线模型及最先进模型相比，该模型在 MPII 和 COCO 数据集上的平均精度均值（mAP）和关键点准确率均有提升。