[Paper Review] ChatGPT as a mapping assistant: A novel method to enrich maps with generative AI and content derived from street-level photographs

Levente Juhász, Peter Mooney+2 more|arXiv (Cornell University)|Jan 1, 2023

Geographic Information Systems Studies10 citations

TL;DR

This paper proposes using ChatGPT as a mapping assistant to enhance OpenStreetMap (OSM) road tagging by leveraging natural language descriptions of Mapillary street-level images. By combining detailed human descriptions with prompt engineering and contextual data (location, detected objects), the method boosts tagging accuracy by up to 29% compared to baseline, demonstrating that generative AI can significantly improve collaborative mapping when guided with rich, structured input.

ABSTRACT

This paper explores the concept of leveraging generative AI as a mapping assistant for enhancing the efficiency of collaborative mapping. We present results of an experiment that combines multiple sources of volunteered geographic information (VGI) and large language models (LLMs). Three analysts described the content of crowdsourced Mapillary street-level photographs taken along roads in a small test area in Miami, Florida. GPT-3.5-turbo was instructed to suggest the most appropriate tagging for each road in OpenStreetMap (OSM). The study also explores the utilization of BLIP-2, a state-of-the-art multimodal pre-training method as an artificial analyst of street-level photographs in addition to human analysts. Results demonstrate two ways to effectively increase the accuracy of mapping suggestions without modifying the underlying AI models: by (1) providing a more detailed description of source photographs, and (2) combining prompt engineering with additional context (e.g. location and objects detected along a road). The first approach increases the suggestion accuracy by up to 29%, and the second one by up to 20%.

Motivation & Objective

• To investigate whether generative AI can accurately suggest OSM road feature tags based on natural language descriptions of street-level images.
• To evaluate the impact of prompt engineering and contextual augmentation (location, detected objects) on the accuracy of AI-generated mapping suggestions.
• To assess the feasibility of using AI as a supportive assistant in collaborative mapping workflows, reducing reliance on human analysts.

Proposed method

• Three human analysts provided detailed textual descriptions of Mapillary street-level images from a Miami test area.
• GPT-3.5-turbo was prompted to suggest OSM 'highway' tags based on these descriptions, using varying levels of prompt complexity.
• BLIP-2 was used to automatically generate image captions and perform visual question answering, serving as an automated analyst alternative.
• Additional context—such as geographic location (e.g., 'near Downtown Miami') and a list of detected objects—was integrated into prompts to improve accuracy.
• Tagging accuracy was evaluated against OSM ground truth, with results analyzed across different prompt configurations and analyst levels.
• The study compared AI suggestions against human-annotated descriptions and assessed additional tagging (e.g., 'lit' tag) for completeness.

Experimental results

Research questions

• RQ1 Can generative AI accurately convert natural language descriptions of street-level images into correct OSM road feature tags?
• RQ2 To what extent can prompt engineering and contextual augmentation (e.g., location, detected objects) improve the accuracy of AI-generated mapping suggestions?
• RQ3 How does the performance of AI-generated suggestions compare to human analysts in a collaborative mapping context?
• RQ4 Can multimodal models like BLIP-2 effectively substitute for human analysts in generating input for LLM-based mapping tasks?

Key findings

• Providing more detailed human descriptions of street-level images increased the accuracy of ChatGPT’s OSM tagging suggestions by up to 29% compared to baseline prompts.
• Incorporating additional context—specifically, the location of the road (e.g., 'near Downtown Miami')—improved accuracy by 4.3% to 8.2% on average.
• Including a list of detected objects in the prompt increased accuracy by 7.5% to 11.0% compared to baseline descriptions.
• Combining both location and object detection context with descriptive prompts boosted accuracy by 12.1% to 19.5% across all analysts.
• ChatGPT correctly suggested the 'lit' tag for 63% to 92% of existing 'lit=yes' cases, and proposed it for an additional 44 to 61 features not yet tagged in OSM.
• BLIP-2-generated captions yielded lower accuracy (27–34%) than human analysts, suggesting that caption quality and detail significantly affect downstream LLM performance.