[论文解读] V-Zero: Self-Improving Multimodal Reasoning with Zero Annotation

Han Wang, Yi Yang+3 more|arXiv (Cornell University)|Jan 15, 2026

Multimodal Machine Learning Applications被引用 0

一句话总结

tldr: V-Zero 引入一个在未标注图像上训练的 Questioner 与 Solver 之间的零注释自我改进循环，在视觉-语言推理方面实现超越人类数据的能力提升。

ABSTRACT

Recent advances in multimodal learning have significantly enhanced the reasoning capabilities of vision-language models (VLMs). However, state-of-the-art approaches rely heavily on large-scale human-annotated datasets, which are costly and time-consuming to acquire. To overcome this limitation, we introduce V-Zero, a general post-training framework that facilitates self-improvement using exclusively unlabeled images. V-Zero establishes a co-evolutionary loop by instantiating two distinct roles: a Questioner and a Solver. The Questioner learns to synthesize high-quality, challenging questions by leveraging a dual-track reasoning reward that contrasts intuitive guesses with reasoned results. The Solver is optimized using pseudo-labels derived from majority voting over its own sampled responses. Both roles are trained iteratively via Group Relative Policy Optimization (GRPO), driving a cycle of mutual enhancement. Remarkably, without a single human annotation, V-Zero achieves consistent performance gains on Qwen2.5-VL-7B-Instruct, improving visual mathematical reasoning by +1.7 and general vision-centric by +2.6, demonstrating the potential of self-improvement in multimodal systems. Code is available at https://github.com/SatonoDia/V-Zero

研究动机与目标

• 目标在于减少对高成本的人类标注数据在多模态推理中的依赖。
• 提出一个在原始图像上运行的零注释后训练框架。
• 设计一个通过内部奖励进行优化的协同进化 Questioner-Solver 循环。
• 证明自生成信号在多样化基准上能够超过有监督基线。

提出的方法

• 从同一个基础大模型中实例化两个角色：Questioner (Q) 与 Solver (S)。
• Q 生成以直觉答案为条件的图像多选题；S 通过对样本进行多数投票给出基于推理的伪标签。
• 使用双轨推理奖励（Dual-Track Reasoning Reward）来鼓励揭示直觉与推理之间差距的问题。
• 在零注释循环中使用组相对策略优化（GRPO）训练 Q 与 S。
• 对 Solver 使用难度引导的数据采样与可验证的强化学习奖励（RLVR）。
• 在 VLMEvalKit 基准上对通用视觉中心任务和数学推理任务进行评估。

Figure 1: Left: Comparison of training paradigms. Supervised GRPO (top) relies on static datasets where rewards are computed by comparing outputs against external ground-truth labels. In contrast, V-Zero (bottom) achieves self-improvement from unlabeled images via a dynamic co-evolutionary loop, whe

实验结果

研究问题

• RQ1 一个视觉语言模型是否能够在没有任何人类标注数据的情况下改进推理能力？
• RQ2 协同进化的 Questioner-Solver 循环是否在有监督基线之上带来可衡量的提升？
• RQ3 双轨奖励和难度引导采样在驱动自我改进方面有多有效？
• RQ4 从几何中心数据的改进是否能迁移到更广义的视觉中心任务？

主要发现

• V-Zero 在没有外部监督的情况下对基础模型持续带来改进。
• 对于 Qwen2.5-VL-7B-Instruct，迭代2的平均分从49.9提升到51.9（+2.0）。
• 在 MMMU 和 MathVerse 上，7B 模型的增益分别达到 +3.9 和 +3.0 点。
• 对于 3B 模型，迭代1 的平均峰值提升为 +0.7 点，且在 MMMU 与 MathVision 上有显著提升。
• V-Zero 在 7B 规模下的有监督 GRPO 基线之上表现优于（平均 51.9 对 50.8）。
• 消融实验表明移除 Questioner、双轨奖励或数据筛选会显著降低性能。

Figure 2: Overview of V-Zero. Our framework drives self-improvement through a co-evolutionary loop. Left (Questioner Training): The Questioner generates a question and an intuitive answer. A frozen Solver then samples reasoning responses to produce a pseudo-label via majority voting. The Questioner