[Paper Review] Multi-skill Mobile Manipulation for Object Rearrangement

Jiayuan Gu, Devendra Singh Chaplot+2 more|arXiv (Cornell University)|Sep 6, 2022

Domain Adaptation and Few-Shot Learning10 citations

TL;DR

This paper proposes M3, a multi-skill mobile manipulation framework that replaces stationary manipulation skills with mobile ones and uses a region-goal navigation reward to improve robustness in long-horizon object rearrangement tasks. By enabling manipulation skills to move and training navigation to target flexible regions rather than fixed points, M3 achieves 71.2% success rate on cross-configuration and 55.0% on cross-layout tasks in the Home Assistant Benchmark, outperforming baselines by significant margins.

ABSTRACT

We study a modular approach to tackle long-horizon mobile manipulation tasks for object rearrangement, which decomposes a full task into a sequence of subtasks. To tackle the entire task, prior work chains multiple stationary manipulation skills with a point-goal navigation skill, which are learned individually on subtasks. Although more effective than monolithic end-to-end RL policies, this framework suffers from compounding errors in skill chaining, e.g., navigating to a bad location where a stationary manipulation skill can not reach its target to manipulate. To this end, we propose that the manipulation skills should include mobility to have flexibility in interacting with the target object from multiple locations and at the same time the navigation skill could have multiple end points which lead to successful manipulation. We operationalize these ideas by implementing mobile manipulation skills rather than stationary ones and training a navigation skill trained with region goal instead of point goal. We evaluate our multi-skill mobile manipulation method M3 on 3 challenging long-horizon mobile manipulation tasks in the Home Assistant Benchmark (HAB), and show superior performance as compared to the baselines.

Motivation & Objective

• To address compounding errors in skill chaining during long-horizon mobile manipulation tasks.
• To improve robustness and composability of modular skills by enabling mobility in manipulation skills.
• To reduce ambiguity in navigation goals by replacing point-goal navigation with region-goal navigation.
• To enhance generalization and success rates in unseen environments (cross-layout) through improved skill formulation.
• To establish design principles—achievable, composable, and reusable skills—for effective subtask decomposition in embodied AI.

Proposed method

• Replace stationary manipulation skills with mobile manipulation skills that allow base movement during task execution.
• Introduce a region-goal navigation reward that defines feasible goal regions instead of fixed point goals, improving navigation robustness.
• Train navigation policies to reach any location within a target region, reducing dependency on precise initial navigation.
• Use collision penalty in the navigation reward to avoid unintended interactions with the environment (e.g., closing doors prematurely).
• Chain mobile manipulation skills with region-goal navigation using a modular skill hierarchy, enabling flexible and error-resilient task execution.
• Formulate initial states of manipulation skills based on a trade-off between achievability and composability, avoiding infeasible or overly broad distributions.

Experimental results

Research questions

• RQ1 Can mobile manipulation skills reduce compounding errors in skill chaining compared to stationary skills?
• RQ2 How does using region-goal navigation instead of point-goal navigation improve policy generalization and success rates?
• RQ3 To what extent can the initial state distribution of manipulation skills be expanded without degrading performance?
• RQ4 Does adding a collision penalty to the navigation reward improve environmental interaction safety and task success?
• RQ5 How do the proposed methods perform under cross-configuration and cross-layout generalization settings?

Key findings

• M3 achieves a 71.2% success rate on the cross-configuration setting and 55.0% on the cross-layout setting in the Home Assistant Benchmark, outperforming all baselines.
• The mobile manipulation approach (M+P) achieves 64.9% success in cross-configuration and 36.2% in cross-layout, significantly outperforming the stationary baseline (S+P) at 57.4% and 31.1% respectively.
• Adding a collision penalty to the navigation reward improves success rates from 57.4% to 65.2% (S+P vs. S+P(C)) and from 64.9% to 67.9% (M+P vs. M+P(C)), demonstrating its importance for safe interaction.
• The ablation study shows that simply expanding the initial state distribution of stationary skills (S(L)+P) reduces performance from 95% to 45% on Pick in TidyHouse, indicating poor feasibility of naive expansion.
• The region-goal navigation reward enables better generalization in cross-layout settings, where M3 achieves 55.0% compared to 36.2% for M+P, highlighting reduced ambiguity in goal specification.
• Qualitative results show that mobile manipulation can correct for imperfect navigation by repositioning to access targets, while stationary manipulation fails when navigation lands in suboptimal zones.