[Paper Review] Learning to Assist: Physics-Grounded Human-Human Control via Multi-Agent Reinforcement Learning

Yuto Shibata, Kashu Yamazaki+4 more|arXiv (Cornell University)|Mar 11, 2026

Social Robot Interaction and HRI0 citations

TL;DR

The paper formulates a multi-agent reinforcement learning framework (AssistMimic) to learn physics-aware, tracking-based controllers for close human–human interactions, enabling an assistant and recipient to co-adapt in a shared physics simulator and achieve assistive motion imitation.

ABSTRACT

Humanoid robotics has strong potential to transform daily service and caregiving applications. Although recent advances in general motion tracking within physics engines (GMT) have enabled virtual characters and humanoid robots to reproduce a broad range of human motions, these behaviors are primarily limited to contact-less social interactions or isolated movements. Assistive scenarios, by contrast, require continuous awareness of a human partner and rapid adaptation to their evolving posture and dynamics. In this paper, we formulate the imitation of closely interacting, force-exchanging human-human motion sequences as a multi-agent reinforcement learning problem. We jointly train partner-aware policies for both the supporter (assistant) agent and the recipient agent in a physics simulator to track assistive motion references. To make this problem tractable, we introduce a partner policies initialization scheme that transfers priors from single-human motion-tracking controllers, greatly improving exploration. We further propose dynamic reference retargeting and contact-promoting reward, which adapt the assistant's reference motion to the recipient's real-time pose and encourage physically meaningful support. We show that AssistMimic is the first method capable of successfully tracking assistive interaction motions on established benchmarks, demonstrating the benefits of a multi-agent RL formulation for physically grounded and socially aware humanoid control.

Motivation & Objective

• Motivate solving assistive and caregiving scenarios that require reactive force exchange between closely interacting humans.
• Develop a MARL framework that jointly trains partner-aware policies for both the supporter and recipient in a physics simulator.
• Leverage an initialization from single-person motion priors to improve exploration and learning efficiency.
• Introduce dynamic reference retargeting and contact-promoting rewards to maintain stable, physically meaningful assistance under noisy references.

Proposed method

• Formulate physics-based human–human imitation as a finite-horizon multi-agent MDP with asymmetric dynamics for two agents (S=Supporter, R=Recipient).
• Extend single-person tracking controllers with partner-aware inputs and an auxiliary assistive state to enable joint learning.
• Initialize policies from a pre-trained single-person motion prior using zero-padding for new inputs to bootstrap learning.
• Implement dynamic reference retargeting to preserve valid relative hand targets when recipients deviate from references.
• Introduce contact-promoting rewards that prioritize active, force-aware interaction over strict kinematic tracking in close-contact scenarios.
• Train specialist policies and distill them into a generalist via DAgger for broader motion coverage.

Figure 2 : Learning contact-rich assistive behaviors is substantially more difficult in the close-contact interactions that we target (bottom) than in contact-less social interactions (top) or isolated motions ( gray SR curve). AssistMimic addresses these challenges, achieving successful imitation f

Experimental results

Research questions

• RQ1 Can joint MARL training of both supporter and recipient learn physically consistent, force-exchanging assistive motions?
• RQ2 Does dynamic reference retargeting improve contact robustness and stability in close-contact assistance?
• RQ3 What is the impact of initializing from single-person motion priors and adding contact-promoting rewards on learning efficiency and imitation fidelity?
• RQ4 How robust are the learned policies to unseen recipient dynamics and generated interaction trajectories?
• RQ5 Does distilling specialist policies into a generalist policy improve generalization to diverse interaction clips?

Key findings

• AssistMimic achieves higher success rates and better stability than baselines in Inter-X (83% SR) and HHI-Assist (66% SR) datasets.
• Joint MARL training outperforms sequential or decoupled learning approaches for closely interacting, force-exchanging motions.
• Motion prior initialization is crucial for convergence; without it, learning fails or exploits rewards.
• Dynamic reference retargeting maintains valid interaction targets and improves robustness, especially on HHI-Assist.
• Contact-promoting rewards significantly enhance robustness to unseen recipient dynamics and improve COM stability under bed-based assistance.
• A generalist policy distilled from specialists via DAgger improves performance on diverse Inter-X clips (SR 64.7% vs 39.8% without distillation).
• The framework can track and reproduce generated diffusion-model interactions, showing broad applicability to generated and unseen motions.

Figure 3 : Overview of AssistMimic . We train tracking-based humanoid control policies for both the recipient and the supporter, optimizing them to imitate a paired reference motion sequence. Our architecture builds on the single-agent tracking framework of PHC [ 10 ] , extending it with partner-awa