[論文レビュー] Unlocking Multimodal Document Intelligence: From Current Triumphs to Future Frontiers of Visual Document Retrieval

Yibo Yan, Jiahao Huo+13 more|arXiv (Cornell University)|Feb 23, 2026

Multimodal Machine Learning Applications被引用数 0

ひとこと要約

この論文は、マルチモーダル大規模言語モデル時代における Visual Document Retrieval (VDR) を概観し、ベンチマーク、手法（エンベディングモデル、リランカー、RAG/エージェント系）、および将来の課題を整理する。

ABSTRACT

With the rapid proliferation of multimodal information, Visual Document Retrieval (VDR) has emerged as a critical frontier in bridging the gap between unstructured visually rich data and precise information acquisition. Unlike traditional natural image retrieval, visual documents exhibit unique characteristics defined by dense textual content, intricate layouts, and fine-grained semantic dependencies. This paper presents the first comprehensive survey of the VDR landscape, specifically through the lens of the Multimodal Large Language Model (MLLM) era. We begin by examining the benchmark landscape, and subsequently dive into the methodological evolution, categorizing approaches into three primary aspects: multimodal embedding models, multimodal reranker models, and the integration of Retrieval-Augmented Generation (RAG) and Agentic systems for complex document intelligence. Finally, we identify persistent challenges and outline promising future directions, aiming to provide a clear roadmap for future multimodal document intelligence.

研究の動機と目的

• Characterize the VDR benchmark landscape and data characteristics in the MLLM era.
• Categorize methodology into embedding models, reranker models, and integration with RAG and agentic systems.
• Identify current challenges in multilingual support, reasoning-intensive retrieval, and efficiency.
• Provide a roadmap of future frontiers for multimodal document intelligence.

提案手法

• Present a formal VDR formulation with query q and document d and late-interaction scoring.
• Review embedding model trends including multi-vector representations and training paradigms.
• Summarize reranker model designs and cross-encoder architectures for fine-grained ranking.
• Explain integration of embedding and reranker in RAG pipelines and agentic systems for document intelligence.
• Discuss training paradigms (pointwise/pairwise/listwise) and objective functions (e.g., InfoNCE).
• Highlight technical innovations across model, data, and training dimensions and efficiency considerations.

実験結果

リサーチクエスチョン

• RQ1 What is the benchmark and data landscape for VDR in the LLM era?
• RQ2 What are the core methodological categories for VDR and how do they evolve with MLLMs?
• RQ3 How do RAG pipelines and agentic systems shape complex document intelligence tasks?
• RQ4 What are the main challenges and future directions for multilingual, reasoning-intensive VDR?

主な発見

• VDR benchmarks have surged recently with datasets spanning thousands to hundreds of thousands of queries and documents.
• Embedding models increasingly use large multimodal language model backbones and multi-vector representations for fine-grained retrieval.
• Reranker models are growing in size and multimodal capabilities but remain largely English-centric except for a few multilingual implementations.
• RAG pipelines and agent-based systems are shifting VDR from static retrieval to dynamic, reasoning-driven workflows.
• Evaluation typically relies on standard IR metrics like nDCG and Recall, with some benchmarks incorporating downstream accuracy and F1 for generation tasks.