[Paper Review] CoFL: Continuous Flow Fields for Language-Conditioned Navigation

Haokun Liu, Zhaoqi MA+5 more|arXiv (Cornell University)|Mar 3, 2026

Multimodal Machine Learning Applications0 citations

TL;DR

CoFL presents an end-to-end policy that maps a Bird’s-Eye View (BEV) image and a language instruction to a continuous flow field, enabling real-time, smooth, obstacle-aware navigation by integrating the field for trajectories. It achieves strong generalization to unseen scenes and zero-shot real-world transfer.

ABSTRACT

Language-conditioned navigation pipelines often rely on brittle modular components or costly action-sequence generation. To address these limitations, we present CoFL, an end-to-end policy that directly maps a bird's-eye view (BEV) observation and a language instruction to a continuous flow field for navigation. Instead of predicting discrete action tokens or sampling action chunks via iterative denoising, CoFL outputs instantaneous velocities that can be queried at arbitrary 2D projected locations. Trajectories are obtained by numerical integration of the predicted field, producing smooth motion that remains reactive under closed-loop execution. To enable large-scale training, we build a dataset of over 500k BEV image-instruction pairs, each procedurally annotated with a flow field and a trajectory derived from BEV semantic maps built on Matterport3D and ScanNet. By training on a mixed distribution, CoFL significantly outperforms modular Vision-Language Model (VLM)-based planners and generative policy baselines on strictly unseen scenes. Finally, we deploy CoFL zero-shot in real-world experiments with overhead BEV observations across multiple layouts, maintaining reliable closed-loop control and a high success rate.

Motivation & Objective

• Motivate language-conditioned navigation that avoids brittle modular pipelines and discrete action tokens.
• Propose a vision-language transformer that predicts a 2D flow field over BEV space conditioned on language.
• Learn from large-scale BEV-image and instruction pairs with procedurally annotated flow fields and trajectories.
• Enable real-time, smooth closed-loop navigation by numerically integrating the predicted field.

Proposed method

• Predict a language-conditioned flow field v(x|I, l) over BEV space from I and l using a transformer-based encoder–decoder.
• Query the flow field at multiple 2D coordinates to produce velocity vectors and form a continuous field.
• Represent velocity as a non-negative magnitude M(x) and unit direction D(x); compute V(x)=M(x)·D(x).
• Train with sampled query points using direction cosine loss and magnitude loss against procedure-generated supervision V*.
• Infer trajectories by forward Euler integration on a dense grid with inverse-time rescaling to maintain quasi-constant velocity.

Figure 2 : Overview of the CoFL’s network architecture. Given a RGB BEV observation $I$ and a language instruction $\ell$ , a SigLIP 2-based [ 39 , 37 ] vision–language encoder produces language-conditioned context tokens over the BEV image. The decoder then queries this context with 2D normalized s

Experimental results

Research questions

• RQ1 Can a single end-to-end model learn a geometry-aware, continuous flow field for language-conditioned navigation from BEV observations?
• RQ2 Does explicit flow-field supervision improve safety (collision avoidance) and trajectory quality over modular or generative baselines in unseen scenes?
• RQ3 Can the model transfer to real-world closed-loop navigation without real-world fine-tuning?
• RQ4 How does grid resolution and inference budgeting affect navigation performance and safety?

Key findings

• CoFL significantly reduces collisions (CR) compared to modular VLM planners and diffusion-policy baselines while maintaining similar final goal error (FGE).
• On Matterport3D, CoFL achieves FGE ~0.13–0.15 with CR ~0.17–0.22 and Curv ~0.08–0.14, outperforming baselines.
• On ScanNet, CoFL improves FGE to ~0.07–0.09 with CR ~0.35–0.40, using a much smaller head (~15M parameters).
• Flow-field predictions are locally accurate (AE/ME) and enable obstacle-aware rollouts at every location, addressing geometric gaps in trajectory-only methods.
• Real-world deployment without fine-tuning shows robust closed-loop control at low latency (about 28 ms per step) across multiple layouts and layouts with static/dynamic obstacles.
• Zero-shot real-world navigation demonstrates reliable performance with a 85% on-target rate without obstacles and 100% on-target with obstacles, while maintaining safe clearance.

Figure 3 : Overview of the trajectory inference. The predicted flow field over the workspace guides agents from different starts toward the same goal while smoothly avoiding obstacles.