[Paper Review] Can LLMs plan paths in the real world?

Wanyi Chen, Meng-Wen Su+2 more|arXiv (Cornell University)|Nov 26, 2024

Private Equity and Venture CapitalBusiness, Management and Accounting3 citations

TL;DR

This study evaluates the real-world path-planning capabilities of three large language models—GPT-4, Gemini, and Mistral—across six diverse real-world scenarios, including urban, suburban, and rural environments. Despite varied prompting strategies, all models produced numerous errors, revealing significant unreliability in practical navigation tasks.

ABSTRACT

As large language models (LLMs) increasingly integrate into vehicle navigation systems, understanding their path-planning capability is crucial. We tested three LLMs through six real-world path-planning scenarios in various settings and with various difficulties. Our experiments showed that all LLMs made numerous errors in all scenarios, revealing that they are unreliable path planners. We suggest that future work focus on implementing mechanisms for reality checks, enhancing model transparency, and developing smaller models.

Motivation & Objective

• To assess whether large language models (LLMs) can reliably generate accurate path-planning instructions in real-world environments.
• To evaluate LLMs across diverse real-world settings, including urban, suburban, and rural areas, with varying levels of complexity.
• To compare performance between turn-by-turn navigation and visual landmark-based navigation (VLN) using LLMs.
• To identify systematic errors in LLM-generated directions and understand their root causes in real-world contexts.
• To advocate for improved model transparency, reality-check mechanisms, and the development of smaller, more reliable models for navigation tasks.

Proposed method

• Conducted six real-world path-planning experiments across three urban, suburban, and rural scenarios involving George Mason University as a starting point.
• Tested three LLMs—GPT-4, Gemini, and Mistral 7B—using both turn-by-turn (TbT) and visual landmark navigation (VLN) prompting strategies.
• Designed prompts to generate step-by-step driving directions or landmark-based navigation cues, simulating real vehicle or pedestrian navigation.
• Collected and analyzed model responses for factual accuracy, route logic, landmark consistency, and temporal planning.
• Performed follow-up queries to test model awareness of real-time data limitations and ability to verify external facts.
• Evaluated model behavior under time constraints and when confronted with contradictory or incorrect information.

(a) GPT’s path

Experimental results

Research questions

• RQ1 Can LLMs generate accurate, safe, and executable turn-by-turn navigation instructions in real-world urban, suburban, and rural environments?
• RQ2 How do LLMs perform in visual landmark-based navigation (VLN) compared to traditional GPS-style routing in complex or sparsely signed environments?
• RQ3 To what extent do LLMs recognize and respond appropriately to real-world constraints such as traffic, time windows, and operational hours?
• RQ4 What types of errors—spatial, factual, or logical—most commonly occur in LLM-generated navigation plans in real-world settings?
• RQ5 How do model size and architecture (e.g., GPT-4 vs. Mistral 7B) influence reliability and consistency in real-world path planning?

Key findings

• All three LLMs—GPT-4, Gemini, and Mistral—produced multiple critical errors in every real-world scenario tested, including incorrect route sequences and false assumptions.
• GPT-4 incorrectly claimed that Shot Tower State Park was in Nevada, later correcting to Virginia after follow-up, indicating a lack of factual grounding.
• Mistral 7B generated a 2,500-mile route from Virginia to Nevada for a park located in Virginia, demonstrating severe spatial reasoning failure.
• LLMs frequently failed to verify real-time operational data, such as park opening hours, and could not access live information to confirm holiday schedules.
• Even with follow-up queries, models often failed to correct errors or acknowledge their own limitations in accessing current data.
• The study concludes that LLMs are currently unreliable for real-world path planning due to hallucinations, poor factual grounding, and lack of reality checks.

(b) Gemini’s path