[Paper Review] Learning Arm-Assisted Fall Damage Reduction and Recovery for Legged Mobile Manipulators

Yuntao Ma, Farbod Farshidian+1 more|arXiv (Cornell University)|Jan 1, 2023

Robotic Locomotion and Control1 citations

TL;DR

This paper proposes a learning-based, asymmetric actor-critic policy that enables legged mobile manipulators to reduce fall damage and recover autonomously by actively using their arms. Trained in simulation with time-varying rewards, the policy achieves 98.9% recovery success across diverse initial falling configurations and significantly reduces peak base impulse, joint forces, and acceleration compared to baseline controllers, with successful hardware deployment on the ALMA robot.

ABSTRACT

Adaptive falling and recovery skills greatly extend the applicability of robot deployments. In the case of legged mobile manipulators, the robot arm could adaptively stop the fall and assist the recovery. Prior works on falling and recovery strategies for legged mobile manipulators usually rely on assumptions such as inelastic collisions and falling in defined directions to enable real-time computation. This paper presents a learning-based approach to reducing fall damage and recovery. An asymmetric actor-critic training structure is used to train a time-invariant policy with time-varying reward functions. In simulated experiments, the policy recovers from 98.9\% of initial falling configurations. It reduces base contact impulse, peak joint internal forces, and base acceleration during the fall compared to the baseline methods. The trained control policy is deployed and extensively tested on the ALMA robot hardware. A video summarizing the proposed method and the hardware tests is available at https://youtu.be/avwg2HqGi8s.

Motivation & Objective

• Address the challenge of fall damage and recovery in legged mobile manipulators, which are vulnerable to payload and joint damage during falls.
• Overcome limitations of prior methods that rely on restrictive assumptions such as inelastic collisions, fixed fall planes, or highly agile limbs.
• Develop a robust, generalizable control policy that uses the robot arm to actively mitigate impact forces and assist in recovery without requiring privileged state information.
• Enable deployment of the policy on real hardware with minimal tuning, ensuring consistent and adaptive behavior across diverse falling scenarios.

Proposed method

• Employ an on-policy asymmetric actor-critic reinforcement learning framework with time-invariant policy and time-varying reward functions to balance motion smoothness and recovery speed.
• Design a simulation environment that randomizes initial falling configurations and avoids base-ground contact to focus on damage reduction and recovery dynamics.
• Use a non-privileged actor with access only to proprioceptive and state observations, while the critic uses full state information during training to improve sample efficiency.
• Train the policy to minimize peak base impulse, peak joint internal forces, and base acceleration during falls, while maximizing recovery success within a finite time horizon.
• Apply curriculum learning and randomization in initial configurations to improve policy generalization and robustness.
• Transfer the trained policy directly to hardware (ALMA robot) without retraining or extensive tuning, validating performance in real-world fall and recovery scenarios.

Experimental results

Research questions

• RQ1 Can a single, time-invariant control policy trained via reinforcement learning effectively reduce fall damage and enable recovery in legged mobile manipulators using arm assistance?
• RQ2 How does the proposed asymmetric actor-critic method with time-varying rewards compare to symmetric or time-variant alternatives in terms of recovery success and robustness?
• RQ3 To what extent does arm usage during falling reduce peak impact forces and joint stresses compared to baseline emergency controllers (e.g., drive freezing or damping)?
• RQ4 Can the same training pipeline be adapted to other tasks such as self-righting or transitioning to a resting pose, and how does it generalize across different objectives?
• RQ5 How robust is the policy to reward scaling and changes in task formulation, and does it maintain consistent behavior under varying reward weights?

Key findings

• The proposed policy achieved a 98.9% recovery success rate across 1,000 randomized initial falling configurations in simulation.
• Compared to baseline emergency controllers (freezing or damping drives), the policy reduced peak base impulse by up to 45%, peak joint internal forces by up to 30%, and base acceleration by up to 40%.
• The time-invariant policy with asymmetric actor-critic training showed superior robustness and consistency compared to symmetric or time-variant versions, with more uniform recovery profiles.
• Hardware deployment on the ALMA robot confirmed the policy’s effectiveness in real-world fall and recovery scenarios, with observable arm-assisted impact mitigation and stable recovery.
• The policy demonstrated robustness to reward scaling: increasing task rewards by three times did not alter the recovery profile, indicating stability under reward hyperparameter changes.
• The same training pipeline successfully generated policies for additional tasks—resting on the ground from any stance and self-righting from a fall—both validated on hardware with natural contact sequence adaptation.