[Paper Review] One Step at a Time: Long-Horizon Vision-and-Language Navigation with Milestones

Chan Hee Song, Jihyung Kil+4 more|arXiv (Cornell University)|Feb 14, 2022

Multimodal Machine Learning Applications4 citations

TL;DR

This paper proposes M-Track, a model-agnostic milestone-based task tracker for long-horizon vision-and-language navigation (VLN) that improves success rates by explicitly monitoring subtask progress. It uses a BERT-CRF milestone builder to extract navigation and interaction milestones from instructions and a milestone checker to validate progress via visual grounding, achieving 33% and 52% relative success rate improvements on ALFRED over strong baselines.

ABSTRACT

We study the problem of developing autonomous agents that can follow human instructions to infer and perform a sequence of actions to complete the underlying task. Significant progress has been made in recent years, especially for tasks with short horizons. However, when it comes to long-horizon tasks with extended sequences of actions, an agent can easily ignore some instructions or get stuck in the middle of the long instructions and eventually fail the task. To address this challenge, we propose a model-agnostic milestone-based task tracker (M-TRACK) to guide the agent and monitor its progress. Specifically, we propose a milestone builder that tags the instructions with navigation and interaction milestones which the agent needs to complete step by step, and a milestone checker that systemically checks the agent's progress in its current milestone and determines when to proceed to the next. On the challenging ALFRED dataset, our M-TRACK leads to a notable 33% and 52% relative improvement in unseen success rate over two competitive base models.

Motivation & Objective

• To address the challenge of long-horizon vision-and-language navigation where agents often skip or stall on subtasks due to poor progress awareness.
• To improve agent performance on complex, multi-step household tasks by introducing explicit milestone-based progress tracking.
• To develop a model-agnostic framework that can be integrated with any VLN agent with minimal architectural changes.
• To reduce subtask failure by proactively rejecting incorrect actions before execution using milestone validation.
• To demonstrate that explicit progress monitoring significantly outperforms implicit monitoring in long-horizon tasks.

Proposed method

• The milestone builder uses a BERT-CRF sequence tagging model to extract target objects and action types (navigation or interaction) from subtask instructions.
• The milestone checker grounds extracted targets in the visual environment using an object detection model and verifies proximity or interaction actions to validate milestone completion.
• The tracker dynamically selects only the current subtask’s instructions for agent input, focusing attention on the active milestone.
• A proactive action rejection mechanism blocks actions that violate milestone constraints (e.g., interacting with the wrong object), preventing subtask failure.
• The system operates solely on language instructions, visual observations, and agent actions, making it agnostic to internal agent architecture.
• The method is evaluated on the ALFRED benchmark, which features long-horizon tasks averaging 50 actions per task.

Figure 1 : Illustration of our M-Track approach . We show an ALFRED task [ 46 ] , which consists of an overall goal (text on the top) and six subtasks (text below each image). The blue / red text box within each image is our extracted navigation / interaction milestones from the subtask instructions

Experimental results

Research questions

• RQ1 Can explicit milestone-based progress tracking improve success rates in long-horizon vision-and-language navigation?
• RQ2 How does milestone-based monitoring compare to implicit progress tracking in complex, multi-step tasks?
• RQ3 To what extent does proactive action rejection based on milestone validation reduce subtask failures?
• RQ4 How accurate is the milestone builder in extracting meaningful navigation and interaction milestones from natural language instructions?
• RQ5 Can the M-Track framework be effectively integrated with diverse VLN agent architectures without architectural modifications?

Key findings

• M-Track achieves a 33% relative improvement in unseen success rate over VLN∘BERT-L on the ALFRED benchmark.
• M-Track achieves a 52% relative improvement in unseen success rate over VLN∘BERT-L on the ALFRED benchmark.
• The milestone checker outperforms a binary classifier baseline for milestone completion detection, indicating the advantage of explicit, environment-grounded validation.
• The upper bound using ground-truth milestones shows only a modest improvement over M-Track’s predicted milestones, suggesting the milestone builder is highly accurate.
• Case studies demonstrate that M-Track prevents subtask skipping and incorrect object interactions, enabling full task completion where the base model fails.
• The model-agnostic design allows seamless integration with existing VLN agents, with minimal changes required to the agent’s input or action pipeline.

Figure 2 : Overview of the milestone checking process . Milestones are extracted from the current low-level instruction by our milestone builder (§ 4.1.1 ). After an action is predicted, our milestone checker (§ 4.1.2 ) examines, based on objects with reachability information (text in images) from i