[Paper Review] MetaMorph: Learning Universal Controllers with Transformers

Agrim Gupta, Linxi Fan+2 more|arXiv (Cornell University)|Mar 22, 2022

Modular Robots and Swarm Intelligence12 citations

TL;DR

MetaMorph proposes a Transformer-based universal controller for modular robots, conditioning policy outputs on robot morphology as a modality. Pre-training across diverse morphologies enables zero-shot generalization to unseen dynamics, kinematics, and tasks, with 2–3× sample efficiency gains in fine-tuning compared to training from scratch.

ABSTRACT

Multiple domains like vision, natural language, and audio are witnessing tremendous progress by leveraging Transformers for large scale pre-training followed by task specific fine tuning. In contrast, in robotics we primarily train a single robot for a single task. However, modular robot systems now allow for the flexible combination of general-purpose building blocks into task optimized morphologies. However, given the exponentially large number of possible robot morphologies, training a controller for each new design is impractical. In this work, we propose MetaMorph, a Transformer based approach to learn a universal controller over a modular robot design space. MetaMorph is based on the insight that robot morphology is just another modality on which we can condition the output of a Transformer. Through extensive experiments we demonstrate that large scale pre-training on a variety of robot morphologies results in policies with combinatorial generalization capabilities, including zero shot generalization to unseen robot morphologies. We further demonstrate that our pre-trained policy can be used for sample-efficient transfer to completely new robot morphologies and tasks.

Motivation & Objective

• Address the challenge of training separate controllers for each robot morphology in modular robotics.
• Enable generalization to unseen robot morphologies, dynamics, and kinematic variations without retraining.
• Achieve sample-efficient transfer to new morphologies and novel tasks using pre-trained policies.
• Explore whether emergent motor synergies in attention mechanisms explain control of high-DOF robots.

Proposed method

• Represent each robot as a sequence of module-level tokens encoding proprioceptive and morphological features.
• Use a Transformer architecture to condition policy outputs on the full robot morphology sequence.
• Pre-train the policy on a diverse distribution of 15–20 DOF modular robots in complex 3D environments.
• Implement dynamic replay buffer balancing to adapt experience collection based on individual robot learning efficiency.
• Analyze attention matrices using stable rank to detect emergent motor synergies in control policies.
• Fine-tune the pre-trained policy on new morphologies and tasks with minimal data, enabling transfer learning.

Experimental results

Research questions

• RQ1 Can a single pre-trained Transformer policy generalize to unseen robot morphologies and dynamics in 3D locomotion tasks?
• RQ2 To what extent does large-scale pre-training improve sample efficiency when transferring to new morphologies and tasks?
• RQ3 Do attention mechanisms in the Transformer policy exhibit signatures of motor synergies, suggesting structured control of high-DOF systems?
• RQ4 How does dynamic replay buffer balancing improve learning efficiency across robots with varying sample complexity?
• RQ5 Can the policy achieve strong zero-shot performance on novel tasks and morphologies not seen during pre-training?

Key findings

• MetaMorph achieves strong zero-shot generalization to unseen robot morphologies, dynamics, and kinematic variations across three complex 3D environments.
• Fine-tuning the pre-trained MetaMorph policy results in 2–3× higher sample efficiency compared to training from scratch on new morphologies.
• The policy generalizes effectively to novel tasks such as escape from a hilly terrain and navigation around cylindrical obstacles, even with distribution shifts.
• Analysis of attention matrices reveals low stable rank and oscillatory patterns, indicating the emergence of motor synergies where groups of limbs are co-activated.
• Dynamic replay buffer balancing improves training efficiency by allocating more experience to slower-learning robots, enhancing overall convergence.
• The pre-trained policy demonstrates robustness to hardware variations, suggesting practical utility in real-world deployment with potential for fault tolerance.