[Paper Review] Mobile ALOHA: Learning Bimanual Mobile Manipulation with Low-Cost Whole-Body Teleoperation

Zipeng Fu, Tony Z. Zhao+1 more|arXiv (Cornell University)|Jan 4, 2024

Robot Manipulation and Learning28 citations

TL;DR

The paper introduces Mobile ALOHA, a low-cost, whole-body teleoperation system for data collection and imitation learning to achieve bimanual mobile manipulation, and shows co-training with static ALOHA data improves performance across several tasks using multiple imitation learning methods.

ABSTRACT

Imitation learning from human demonstrations has shown impressive performance in robotics. However, most results focus on table-top manipulation, lacking the mobility and dexterity necessary for generally useful tasks. In this work, we develop a system for imitating mobile manipulation tasks that are bimanual and require whole-body control. We first present Mobile ALOHA, a low-cost and whole-body teleoperation system for data collection. It augments the ALOHA system with a mobile base, and a whole-body teleoperation interface. Using data collected with Mobile ALOHA, we then perform supervised behavior cloning and find that co-training with existing static ALOHA datasets boosts performance on mobile manipulation tasks. With 50 demonstrations for each task, co-training can increase success rates by up to 90%, allowing Mobile ALOHA to autonomously complete complex mobile manipulation tasks such as sauteing and serving a piece of shrimp, opening a two-door wall cabinet to store heavy cooking pots, calling and entering an elevator, and lightly rinsing a used pan using a kitchen faucet. Project website: https://mobile-aloha.github.io

Motivation & Objective

• Demonstrate a low-cost, whole-body teleoperation system for collecting bimanual mobile manipulation data.
• Show that concatenating base and arm actions enables effective imitation learning for mobile manipulation.
• Investigate co-training with static ALOHA datasets to improve data efficiency and performance.
• Evaluate compatibility with multiple imitation learning methods across diverse tasks.

Proposed method

• Extend the ALOHA system with a wheeled mobile base to enable full-body teleoperation and collect joint-arm and base velocity data.
• Formulate actions as a 16D vector combining 14-DoF arm joint positions and 2D base velocities for end-to-end imitation learning.
• Apply co-training by mixing Mobile ALOHA demonstrations with static ALOHA data without special preprocessing, training with a shared imitation loss.
• Evaluate compatibility of Mobile ALOHA data with ACT, Diffusion Policy, and VINN, using action chunking to improve trajectory coherence.
• Use equal-probability data sampling from mobile and static datasets and zero-pad static actions to match dimensions for training.
• Conduct extensive real-world experiments on seven mobile manipulation tasks to assess success rates and data efficiency.

Experimental results

Research questions

• RQ1 Can Mobile ALOHA acquire complex bimanual mobile manipulation skills with a small number of demonstrations using co-training?
• RQ2 Do existing imitation learning methods (ACT, Diffusion Policy, VINN) work effectively with Mobile ALOHA data?
• RQ3 Does co-training with static bimanual data improve performance and data efficiency for mobile manipulation tasks?
• RQ4 What is the impact of data mixture proportions and pre-training on performance?
• RQ5 What are the practical data collection and teleoperation ergonomics for learning mobile manipulation?

Key findings

• Co-training with static ALOHA data improves success rates across multiple tasks, achieving up to 90% improvements with 50 demonstrations per task.
• Mobile ALOHA enables high-performance bimanual mobile manipulation for tasks like opening cabinets, calling an elevator, and cooking actions with 80-95% success on several tasks.
• ACT, Diffusion Policy, and VINN are compatible with Mobile ALOHA data; ACT and Diffusion Policy achieve strong performance with co-training, while VINN shows mixed results.
• Co-training improves data efficiency; with 35 in-domain demonstrations, co-training can outperform a 50-demo non-co-training policy.
• Co-training is robust to different data mixtures and outperforms pre-training on static ALOHA data for the evaluated tasks.