[Paper Review] Minimizing Energy Consumption Leads to the Emergence of Gaits in Legged Robots

Zipeng Fu, Ashish Kumar+2 more|arXiv (Cornell University)|Oct 25, 2021

Robotic Locomotion and Control41 citations

TL;DR

The paper shows that learning to minimize energy leads to emergent gaits (walk, trot, bounce) in a quadruped on flat terrain and unstructured, irregular gaits on rough terrain, validated in simulation and on real hardware.

ABSTRACT

Legged locomotion is commonly studied and expressed as a discrete set of gait patterns, like walk, trot, gallop, which are usually treated as given and pre-programmed in legged robots for efficient locomotion at different speeds. However, fixing a set of pre-programmed gaits limits the generality of locomotion. Recent animal motor studies show that these conventional gaits are only prevalent in ideal flat terrain conditions while real-world locomotion is unstructured and more like bouts of intermittent steps. What principles could lead to both structured and unstructured patterns across mammals and how to synthesize them in robots? In this work, we take an analysis-by-synthesis approach and learn to move by minimizing mechanical energy. We demonstrate that learning to minimize energy consumption plays a key role in the emergence of natural locomotion gaits at different speeds in real quadruped robots. The emergent gaits are structured in ideal terrains and look similar to that of horses and sheep. The same approach leads to unstructured gaits in rough terrains which is consistent with the findings in animal motor control. We validate our hypothesis in both simulation and real hardware across natural terrains. Videos at https://energy-locomotion.github.io

Motivation & Objective

• Motivate a shift from pre-programmed gait libraries to energy-driven gait emergence in legged robots.
• Demonstrate that energy minimization can produce structured gaits at different speeds on flat terrain and unstructured gaits on uneven terrain.
• Show sim-to-real transfer of energy-driven policies to a real quadruped robot.
• Provide a velocity-conditioned policy to enable smooth gait transitions across speeds.

Proposed method

• Use an end-to-end, model-free reinforcement learning framework to learn joint-angle actions that minimize energy while moving forward.
• Define the policy as a multilayer perceptron taking state (30D) and previous action (12D) to predict 12 joint target angles, converted to torque via a PD controller.
• Adopt a bio-energetics-based reward: r = r_forward + alpha1 * r_energy + r_alive, with r_energy = -tau^T qdot.
• Train on fractal terrains to encourage robust foot clearance and prevent reliance on artificial penalties.
• Transfer policies to a real robot using Rapid Motor Adaptation (RMA) for sim-to-real adaptation of extrinsics.
• Employ a velocity-conditioned learning scheme with distillation from expert, velocity-specific policies to enable smooth gait transitions.

Experimental results

Research questions

• RQ1 Can energy minimization alone yield natural, gait-like patterns at different speeds without pre-programmed gaits?
• RQ2 Do emergent gaits on flat terrain correspond to known animal gaits and Froude-number ranges found in livestock and horses?
• RQ3 Can a velocity-conditioned policy achieve smooth transitions between emergent gaits as target speed changes?
• RQ4 Is sim-to-real transfer viable for energy-efficient, emergent gait policies on a real quadruped across diverse terrains?

Key findings

• Emergent gaits on flat terrains include walk, trot, and bounce as speed increases, with energy efficiency guiding gait selection.
• The emergent gaits at corresponding speeds align with Froude-number-based similarities to sheep and horses, without any gait pre-programming.
• On uneven terrains, the same framework yields unstructured, irregular gaits consistent with natural animal locomotion.
• Real-world deployments show speeds closely matching target speeds (e.g., 0.375, 0.9, 1.5 m/s targets with corresponding actuals) and energy-efficient performance exceeding a convex MPC baseline.
• A velocity-conditioned policy with distillation from expert gait policies enables smooth transitions across a continuous speed range, demonstrating robust sim-to-real transfer.