[论文解读] VLRS-Bench: A Vision-Language Reasoning Benchmark for Remote Sensing

Zhiming Luo, Di Wang+3 more|arXiv (Cornell University)|Feb 4, 2026

Multimodal Machine Learning Applications被引用 0

一句话总结

VLRS-Bench 是首个聚焦于遥感领域复杂视觉-语言推理的基准测试，围绕 Cognition、Decision、Prediction 结构化评估 MLLMs 的地理空间推理与预测能力。

ABSTRACT

Recent advancements in Multimodal Large Language Models (MLLMs) have enabled complex reasoning. However, existing remote sensing (RS) benchmarks remain heavily biased toward perception tasks, such as object recognition and scene classification. This limitation hinders the development of MLLMs for cognitively demanding RS applications. To address this, , we propose a Vision Language ReaSoning Benchmark (VLRS-Bench), which is the first benchmark exclusively dedicated to complex RS reasoning. Structured across the three core dimensions of Cognition, Decision, and Prediction, VLRS-Bench comprises 2,000 question-answer pairs with an average length of 71 words, spanning 14 tasks and up to eight temporal phases. VLRS-Bench is constructed via a specialized pipeline that integrates RS-specific priors and expert knowledge to ensure geospatial realism and reasoning complexity. Experimental results reveal significant bottlenecks in existing state-of-the-art MLLMs, providing critical insights for advancing multimodal reasoning within the remote sensing community.

研究动机与目标

• Motivate and quantify the need for cognition-driven, domain-aware multimodal reasoning in remote sensing (RS).
• Provide a hierarchically structured benchmark (Cognition, Decision, Prediction) to assess higher-order RS reasoning tasks.
• Incorporate RS priors (DSM, NIR, expert masks) and multi-temporal data to ensure geospatial realism in tasks.
• Establish an automated, RS-tailored pipeline to generate and verify challenging reasoning tasks with expert grounding.

提出的方法

• Define a three-level reasoning taxonomy (Cognition, Decision, Prediction) with six L-2 abilities and fourteen L-3 tasks.
• Automated pipeline that fuses RGB RS imagery with RS priors (DSM, NIR), expert masks, and multi-temporal references to create multimodal instructions.
• Use GPT-5-chat to generate QA items, then convert them into multiple formats (MCQ, true/false, fill-in-the-blank).
• Three-stage verification: automated filtering, multi-MLLM cross-validation, and human expert review to ensure task quality and grounding.
• Evaluate a wide range of MLLMs (general, RS-specialized) in zero-shot setting with standardized prompts.
• Report per-dimension and per-task performance to diagnose bottlenecks in cognition, planning, and temporal forecasting.

Figure 1 : Pipeline for constructing VLRS-Bench. The process integrates the target RGB image with multi-source remote sensing priors ( e.g . , DSM and expert masks) to form a structured multimodal instruction, which guides a GPT-5-chat to produce reasoning tasks across cognitive dimensions. Each gen

实验结果

研究问题

• RQ1 Can current MLLMs perform genuine geospatial cognition beyond static perception in RS scenarios?
• RQ2 How do model capabilities differ across Cognition, Decision, and Prediction facets in RS reasoning?
• RQ3 What is the impact of RS priors (DSM, NIR, masks) and multi-temporal references on reasoning realism and task difficulty?
• RQ4 Do RS-specific MLLMs outperform general-purpose MLLMs on complex RS reasoning tasks, and where do gaps remain?

主要发现

• General MLLMs show weaker temporal-spatiotemporal reasoning compared with static cognition.
• RS-specialized MLLMs outperform larger general models in several reasoning aspects but struggle with complex decision-making and long-horizon prediction.
• Semantic integration tasks are more tractable for current models than mechanistic interaction reasoning.
• Model performance declines as answer space becomes more complex (multi-choice, fill-in-the-blank).
• Decision tasks improve with model scale, but planning and evaluation can be decoupled (PR vs ER).
• Prediction tasks reveal increasing difficulty from local object-level forecasts to global scene evolution and higher sensitivity to uncertainty.

Figure 2 : Avg. Score of various MLLMs across four QA-types. The distinct color coding ( e.g . Qwen2.5-VL-32B in Blue , GPT-4o-mini in Yellow ) highlights a critical phenomenon: a sharp performance drop from Single-Choice to Multi-Choice and Fill in Blank tasks. This trend, consistent across model s