[Paper Review] How to Train PointGoal Navigation Agents on a (Sample and Compute) Budget

Erik Wijmans, Irfan Essa+1 more|arXiv (Cornell University)|Dec 10, 2020

Multimodal Machine Learning Applications4 citations

TL;DR

This paper presents a set of efficient training strategies for PointGoal navigation agents under strict sample (75M frames) and compute (1 GPU, 1 day) budgets. By optimizing advantage estimation, visual encoder architecture, and a key hyperparameter, the authors achieve 8 SPL and 20 SPL gains on Gibson and Matterport3D respectively under sample budget, and 19 SPL and 35 SPL improvements under compute budget—demonstrating substantial performance gains with minimal resources.

ABSTRACT

PointGoal navigation has seen significant recent interest and progress, spurred on by the Habitat platform and associated challenge. In this paper, we study PointGoal navigation under both a sample budget (75 million frames) and a compute budget (1 GPU for 1 day). We conduct an extensive set of experiments, cumulatively totaling over 50,000 GPU-hours, that let us identify and discuss a number of ostensibly minor but significant design choices -- the advantage estimation procedure (a key component in training), visual encoder architecture, and a seemingly minor hyper-parameter change. Overall, these design choices to lead considerable and consistent improvements over the baselines present in Savva et al. Under a sample budget, performance for RGB-D agents improves 8 SPL on Gibson (14% relative improvement) and 20 SPL on Matterport3D (38% relative improvement). Under a compute budget, performance for RGB-D agents improves by 19 SPL on Gibson (32% relative improvement) and 35 SPL on Matterport3D (220% relative improvement). We hope our findings and recommendations will make serve to make the community's experiments more efficient.

Motivation & Objective

• To improve PointGoal navigation performance under strict sample and compute constraints.
• To identify and optimize seemingly minor design choices that significantly impact training efficiency and final performance.
• To provide actionable, resource-efficient training recipes for the broader research community.

Proposed method

• Employing a sample budget of 75 million frames and a compute budget of 1 GPU for 1 day, the authors conduct extensive ablation studies.
• Optimizing the advantage estimation procedure to improve credit assignment during policy learning.
• Evaluating different visual encoder architectures to identify those most effective under resource constraints.
• Tuning a key hyperparameter related to the value function loss to enhance training stability and performance.
• Using the Habitat simulation platform to train and evaluate RGB-D agents on Gibson and Matterport3D environments.
• Conducting over 50,000 GPU-hours of experiments to systematically assess design choices.

Experimental results

Research questions

• RQ1 How do different advantage estimation methods affect navigation performance under sample and compute limits?
• RQ2 Which visual encoder architectures yield the best performance-to-resource ratio in PointGoal navigation?
• RQ3 What impact does a specific hyperparameter change in the value function loss have on final agent performance?
• RQ4 Can significant performance gains be achieved without increasing compute or data budgets?
• RQ5 What combination of design choices leads to the most efficient and effective training of PointGoal navigation agents?

Key findings

• Under a sample budget, RGB-D agents achieve an 8 SPL improvement on Gibson (14% relative gain) and a 20 SPL improvement on Matterport3D (38% relative gain) over baseline methods.
• Under a compute budget, RGB-D agents achieve a 19 SPL improvement on Gibson (32% relative gain) and a 35 SPL improvement on Matterport3D (220% relative gain).
• The choice of advantage estimation procedure significantly impacts learning efficiency and final performance, with the best variant yielding consistent gains.
• Optimizing the visual encoder architecture leads to measurable performance improvements, especially under compute constraints.
• A seemingly minor hyperparameter change in the value function loss results in substantial and consistent performance gains across both datasets.
• The combination of these design choices enables state-of-the-art performance with minimal computational and data resources.