[Paper Review] GenLoco: Generalized Locomotion Controllers for Quadrupedal Robots

Gilbert Feng, Hongbo Zhang+9 more|arXiv (Cornell University)|Sep 12, 2022

Robotic Locomotion and Control17 citations

TL;DR

This paper introduces GenLoco, a reinforcement learning framework that trains generalized locomotion controllers for quadrupedal robots by randomizing robot morphology during simulation. By training on a diverse set of procedurally generated quadrupedal robots with varying sizes, masses, and leg lengths, the resulting policies achieve zero-shot transfer to unseen real-world robots—including Unitree A1, MIT Mini Cheetah, and CUHK Sirius—outperforming robot-specific policies in both simulation and real-world deployment.

ABSTRACT

Recent years have seen a surge in commercially-available and affordable quadrupedal robots, with many of these platforms being actively used in research and industry. As the availability of legged robots grows, so does the need for controllers that enable these robots to perform useful skills. However, most learning-based frameworks for controller development focus on training robot-specific controllers, a process that needs to be repeated for every new robot. In this work, we introduce a framework for training generalized locomotion (GenLoco) controllers for quadrupedal robots. Our framework synthesizes general-purpose locomotion controllers that can be deployed on a large variety of quadrupedal robots with similar morphologies. We present a simple but effective morphology randomization method that procedurally generates a diverse set of simulated robots for training. We show that by training a controller on this large set of simulated robots, our models acquire more general control strategies that can be directly transferred to novel simulated and real-world robots with diverse morphologies, which were not observed during training.

Motivation & Objective

• To address the high cost of training robot-specific locomotion controllers for each new quadrupedal platform.
• To enable transferable, general-purpose locomotion policies that work across robots with different morphologies and dynamics.
• To reduce reliance on precise simulation-to-reality sim-to-real transfer by training on diverse morphologies during policy learning.
• To demonstrate that a single policy can be deployed across multiple real-world quadrupedal robots without fine-tuning.
• To provide a reusable, open-source baseline for new quadrupedal robots, avoiding the need for costly retraining.

Proposed method

• Procedural generation of diverse simulated quadrupedal robots by randomizing body size, leg length, and mass within a morphological template.
• Training a history-based deep reinforcement learning policy on this diverse set of simulated robots to learn general control strategies.
• Applying morphology randomization during training to expose the policy to a wide range of kinematic and dynamic variations.
• Using a single policy head with shared parameters across all simulated morphologies to encourage generalization.
• Evaluating transfer performance via zero-shot deployment on real-world robots not seen during training.
• Maintaining a fixed number of degrees of freedom (DoFs) and link structure across all simulated robots to ensure architectural consistency.

Experimental results

Research questions

• RQ1 Can a single locomotion policy be trained to generalize across quadrupedal robots with diverse morphologies?
• RQ2 Does morphology randomization during training improve sim-to-real transfer performance compared to training on a single robot?
• RQ3 Can a policy trained on simulated robots be deployed directly on real-world robots without fine-tuning?
• RQ4 How does the performance of a generalized policy compare to robot-specific policies in real-world deployment?
• RQ5 To what extent does morphology randomization improve robustness to out-of-distribution dynamics changes?

Key findings

• GenLoco policies achieved a normalized return of 0.773 ± 0.054 on the Unitree A1 and 0.743 ± 0.092 on the Mini Cheetah during pacing gait, outperforming A1-specific policies (0.696 ± 0.032) and Cheetah-specific policies (0 in 6 trials).
• For spinning gaits, GenLoco achieved 0.670 ± 0.070 on A1 and 0.721 ± 0.090 on Cheetah, outperforming A1-specific (0.572 ± 0.049) and Cheetah-specific (0.258 ± 0.013) policies.
• The Mini Cheetah-specific policy failed in all 6 real-world trials despite working in simulation, while GenLoco policies succeeded consistently, demonstrating superior sim-to-real robustness.
• GenLoco policies showed notable robustness to changes in motor friction and strength, indicating effective generalization to out-of-distribution dynamics.
• Deployment on the newly developed CUHK Sirius robot was achieved within days of hardware availability, confirming zero-shot transfer capability.
• Performance degradation was observed for larger robots, suggesting limitations in generalization scope under extreme morphological variations.