[论文解读] Learning to Assist: Physics-Grounded Human-Human Control via Multi-Agent Reinforcement Learning

Yuto Shibata, Kashu Yamazaki+4 more|arXiv (Cornell University)|Mar 11, 2026

Social Robot Interaction and HRI被引用 0

一句话总结

本文提出一个多智能体强化学习框架（AssistMimic），学习物理感知、基于跟踪的控制器用于近距离的人–人交互协作，使助理与接受者在共享物理仿真器中共同自适应并实现协助性动作模仿。

ABSTRACT

Humanoid robotics has strong potential to transform daily service and caregiving applications. Although recent advances in general motion tracking within physics engines (GMT) have enabled virtual characters and humanoid robots to reproduce a broad range of human motions, these behaviors are primarily limited to contact-less social interactions or isolated movements. Assistive scenarios, by contrast, require continuous awareness of a human partner and rapid adaptation to their evolving posture and dynamics. In this paper, we formulate the imitation of closely interacting, force-exchanging human-human motion sequences as a multi-agent reinforcement learning problem. We jointly train partner-aware policies for both the supporter (assistant) agent and the recipient agent in a physics simulator to track assistive motion references. To make this problem tractable, we introduce a partner policies initialization scheme that transfers priors from single-human motion-tracking controllers, greatly improving exploration. We further propose dynamic reference retargeting and contact-promoting reward, which adapt the assistant's reference motion to the recipient's real-time pose and encourage physically meaningful support. We show that AssistMimic is the first method capable of successfully tracking assistive interaction motions on established benchmarks, demonstrating the benefits of a multi-agent RL formulation for physically grounded and socially aware humanoid control.

研究动机与目标

• 推动解决需要紧密互动人之间的反应性力交换的协助与照护场景。
• 开发一个 MARL 框架，在物理仿真器中共同训练提供者与接受者的伙伴感知策略。
• 利用单人运动先验初始化以提升探索与学习效率。
• 引入动态参考重定向和接触促进奖励，在有噪声的参考下保持稳定、物理意义明确的协助。

提出的方法

• 将基于物理的人-人模仿问题形式化为一个有限时域的多智能体MDP，对于两个智能体（S=Supporter，R=Recipient）具有非对称动力学。
• 在单人跟踪控制器上扩展伙伴感知输入和辅助状态，以实现联合学习。
• 使用零填充新输入，将来自预训练的单人运动先验初始化策略以引导学习。
• 实现动态参考重定向，在接受者偏离参考时保持有效的相对手部目标。
• 引入接触促进奖励，在近距离场景中优先考虑主动、受力感知的交互而非严格的运动学跟踪。
• 通过 DAgger 将专家策略蒸馏为通用策略以实现更广的运动覆盖。

Figure 2 : Learning contact-rich assistive behaviors is substantially more difficult in the close-contact interactions that we target (bottom) than in contact-less social interactions (top) or isolated motions ( gray SR curve). AssistMimic addresses these challenges, achieving successful imitation f

实验结果

研究问题

• RQ1 两者均衡 MARL 训练的提供者与接受者能否学习到物理一致、具备力交换的协助性动作？
• RQ2 动态参考重定向是否提升近距离协助中的接触鲁棒性与稳定性？
• RQ3 以单人运动先验初始化并加入接触促进奖励对学习效率与模仿保真度有何影响？
• RQ4 学习到的策略对未见的接受者动力学和生成的交互轨迹有多鲁棒？
• RQ5 通过 DAgger 将专家策略蒸馏成通用策略是否提升对多样化交互片段的泛化能力？

主要发现

• AssistMimic 在 Inter-X 数据集（83% SR）和 HHI-Assist 数据集（66% SR）上实现更高的成功率与更好的稳定性，相较于基线。
• 联合 MARL 训练在紧密交互、具力交换的运动中优于顺序或解耦学习方法。
• 运动先验初始化对于收敛至关重要；没有它，学习要么失败要么利用奖励进行异常 optimize。
• 动态参考重定向保持有效的交互目标并提升鲁棒性，特别是在 HHI-Assist 上表现更好。
• 接触促进奖励显著提升对未见接受者动力学的鲁棒性，并在床上辅助情景下改善质心（COM）稳定性。
• 通过 DAgger 将专家策略蒸馏成通用策略后，在多样的 Inter-X 片段上性能提升（SR 从 39.8% 提升至 64.7%）。
• 该框架能够跟踪并重现生成的扩散模型交互，显示对生成与未见运动的广泛适用性。

Figure 3 : Overview of AssistMimic . We train tracking-based humanoid control policies for both the recipient and the supporter, optimizing them to imitate a paired reference motion sequence. Our architecture builds on the single-agent tracking framework of PHC [ 10 ] , extending it with partner-awa