[Paper Review] MINOS: Multimodal Indoor Simulator for Navigation in Complex Environments

Manolis Savva, Anne Lynn S. Chang+3 more|arXiv (Cornell University)|Dec 11, 2017

Mobile Crowdsensing and CrowdsourcingComputer Science13 references151 citations

TL;DR

MINOS provides a large-scale multimodal indoor simulation framework (SUNCG and Matterport3D) to develop and benchmark goal-directed navigation in realistic cluttered environments, evaluating multimodal sensor inputs and multiple RL-based navigation methods.

ABSTRACT

We present MINOS, a simulator designed to support the development of multisensory models for goal-directed navigation in complex indoor environments. The simulator leverages large datasets of complex 3D environments and supports flexible configuration of multimodal sensor suites. We use MINOS to benchmark deep-learning-based navigation methods, to analyze the influence of environmental complexity on navigation performance, and to carry out a controlled study of multimodality in sensorimotor learning. The experiments show that current deep reinforcement learning approaches fail in large realistic environments. The experiments also indicate that multimodality is beneficial in learning to navigate cluttered scenes. MINOS is released open-source to the research community at http://minosworld.org . A video that shows MINOS can be found at https://youtu.be/c0mL9K64q84

Motivation & Objective

• Motivate and enable development of multisensory navigation models in realistic indoor environments.
• Leverage large datasets (SUNCG and Matterport3D) to study generalization across diverse layouts.
• Provide flexible multimodal sensor support and environment customization for controlled experiments.
• Benchmark state-of-the-art RL-based navigation methods and analyze the impact of environmental complexity on performance.
• Demonstrate the benefit of multimodal inputs for cluttered indoor navigation and release the tool open-source.

Proposed method

• Introduce MINOS as a fast, flexible simulator with datasets SUNCG (≈45k houses) and Matterport3D (≈90 residences).
• Support modular sensor suites (vision, depth, surface normals, touch, semantics) and programmable sensor configurations.
• Provide environment customization (texture/ clutter variation, object manipulation) and goal-driven tasks (PointGoal, ObjectGoal, RoomGoal).
• Implement a server-client architecture with WebGL rendering and Python/web clients for RL training and interactive use.
• Allow both continuous and discrete agent control, with a parameterized physics model and multiple pre-configured agents (discrete/continuous).
• Benchmark multiple agents (A3C FF, A3C LSTM, UNREAL, Direct Future Prediction) under varied modalities and environmental complexities.

Experimental results

Research questions

• RQ1 How do current deep RL navigation methods perform in large-scale, realistic indoor environments?
• RQ2 Does multimodal sensing (vision, depth, touch, semantics) improve navigation performance in cluttered indoor scenes?
• RQ3 How does environmental complexity (size, furniture, realism) affect generalization to unseen environments?
• RQ4 What is the role of dataset realism (synthetic SUNCG vs reconstructed Matterport3D) in navigation performance?
• RQ5 Can MINOS facilitate controlled comparisons across methods and sensor configurations?

Key findings

• Most deep RL methods struggle in furnished Matterport3D environments, with PointGoal success often under 20%.
• RoomGoal tasks show even lower performance, highlighting sparser reward signals in semantic goals.
• UNREAL generally performs best among the evaluated methods, followed by DFP and A3C-LSTM; A3C-FF performs poorly.
• Depth and touch modalities are especially powerful; combining modalities yields the best performance in cluttered environments.
• Multimodal inputs significantly improve navigation in cluttered settings; depth (and combinations including depth) frequently provide the strongest gains.
• Performance degrades substantially as environment size and clutter increase, indicating current RL methods have limited scalability in realistic indoor navigation.