[论文解读] UniFinEval: Towards Unified Evaluation of Financial Multimodal Models across Text, Images and Videos

Zhi Yang, Lingfeng Zeng+17 more|arXiv (Cornell University)|Jan 9, 2026

Stock Market Forecasting Methods被引用 0

一句话总结

UniFinEval 是一个人工构建的双语（中英）多模态基准，评估金融大语言模型在文本、图像和视频等五个核心场景中的跨模态多步推理能力。它在零-shot 与零-shot 连续思维（Zero-Shot CoT）设置下比较了10个主流模型，揭示面向金融专家的仍存差距。

ABSTRACT

Multimodal large language models are playing an increasingly significant role in empowering the financial domain, however, the challenges they face, such as multimodal and high-density information and cross-modal multi-hop reasoning, go beyond the evaluation scope of existing multimodal benchmarks. To address this gap, we propose UniFinEval, the first unified multimodal benchmark designed for high-information-density financial environments, covering text, images, and videos. UniFinEval systematically constructs five core financial scenarios grounded in real-world financial systems: Financial Statement Auditing, Company Fundamental Reasoning, Industry Trend Insights, Financial Risk Sensing, and Asset Allocation Analysis. We manually construct a high-quality dataset consisting of 3,767 question-answer pairs in both chinese and english and systematically evaluate 10 mainstream MLLMs under Zero-Shot and CoT settings. Results show that Gemini-3-pro-preview achieves the best overall performance, yet still exhibits a substantial gap compared to financial experts. Further error analysis reveals systematic deficiencies in current models. UniFinEval aims to provide a systematic assessment of MLLMs' capabilities in fine-grained, high-information-density financial environments, thereby enhancing the robustness of MLLMs applications in real-world financial scenarios. Data and code are available at https://github.com/aifinlab/UniFinEval.

研究动机与目标

• 评估多模态大语言模型（MLLMs）在高信息密度金融环境中的能力边界。
• 提供与真实世界金融工作流对齐的统一跨模态基准。
• 使评估跨模态一致性和金融领域的多步推理成为可能。
• 识别常见的失败模式，为稳健的金融AI部署提供指引。

提出的方法

• 人工构建的3,767问题的中英双语数据集。
• 五个金融场景：财务报表审计、公司基本面推理、行业趋势洞察、金融风险感知、资产配置分析。
• 支持完整模态输入：文本、图像和视频及其跨模态组合（文本-图像、文本-视频、图像-视频、文本-图像-视频）。
• 两种评估设置：零-shot 与零-shot 连续思维，输出提取由Qwen-Max标准化，以实现鲁棒判定。
• 通过专家驱动的质量控制，四阶段验证以确保与真实金融逻辑的一致性。

Figure 1: UniFinEval is manually constructed and supports full-modality inputs including text, images, and videos. It is equipped with cross-modal reasoning capabilities and features high information density while closely aligning with real financial business practices.

实验结果

研究问题

• RQ1 当前的MLLMs在高信息密度金融任务中是否能够实现跨模态的综合推理？
• RQ2 现有模型在感知、推理和决策任务上离金融专家的表现有多接近？
• RQ3 在处理多模态金融信息时，主要的错误模式是什么？
• RQ4 链式思维提示在金融特定跨模态任务中的性能影响如何？
• RQ5 当前基准在模拟真实金融决策循环方面有哪些局限性？

主要发现

• Gemini-3-pro-preview 在零-shot 下的综合表现最佳，平均为 73.8%。
• 大多数模型在 CoT 下有提升，但在不同任务上的增益有限。
• 人类（专家）显著优于所有模型，在 ITI 与 AAA 场景存在较大差距。
• 错误分析显示图像感知与跨模态对齐存在主要问题，数值计算能力也存在显著不足。
• 模型在高信息密度任务中难以完成跨模态多步推理以及长期逻辑一致性的维护。

Figure 2: UniFinEval covers five major financial scenarios and constructs datasets spanning text, images, videos, as well as multiple cross-modal combinations. It features high-information-density and manually construct data, together with dedicated designs for cross-modal consistency checking and m