[论文解读] RPIQ: Residual-Projected Multi-Collaboration Closed-Loop and Single Instance Quantization for Visually Impaired Assistance

Xuanyu Wang, Haisen Su+10 more|arXiv (Cornell University)|Jan 6, 2026

Multimodal Machine Learning Applications被引用 0

一句话总结

RPIQ 引入了一种分块、多轮残差补偿量化框架，结合单实例 Hessian 基线校准，实现对大模型的 4 位量化，在显著减少内存的同时，保持对视觉障碍辅助任务的性能。

ABSTRACT

Visually impaired users face significant challenges in daily information access and real-time environmental perception, and there is an urgent need for intelligent assistive systems with accurate recognition capabilities. Although large-scale models provide effective solutions for perception and reasoning, their practical deployment on assistive devices is severely constrained by excessive memory consumption and high inference costs. Moreover, existing quantization strategies often ignore inter-block error accumulation, leading to degraded model stability. To address these challenges, this study proposes a novel quantization framework -- Residual-Projected Multi-Collaboration Closed-Loop and Single Instance Quantization(RPIQ), whose quantization process adopts a multi-collaborative closed-loop compensation scheme based on Single Instance Calibration and Gauss-Seidel Iterative Quantization. Experiments on various types of large-scale models, including language models such as OPT, Qwen, and LLaMA, as well as vision-language models such as CogVLM2, demonstrate that RPIQ can compress models to 4-bit representation while significantly reducing peak memory consumption (approximately 60%-75% reduction compared to original full-precision models). The method maintains performance highly close to full-precision models across multiple language and visual tasks, and exhibits excellent recognition and reasoning capabilities in key applications such as text understanding and visual question answering in complex scenarios. While verifying the effectiveness of RPIQ for deployment in real assistive systems, this study also advances the computational efficiency and reliability of large models, enabling them to provide visually impaired users with the required information accurately and rapidly.

研究动机与目标

• 提高在视觉障碍辅助任务中使用的大模型的量化稳定性和准确性。
• 缓解分块 GPTQ 风格量化中固有的分块间误差累积。
• 在量化过程中减少校准数据依赖和内存占用。
• 在资源受限的辅助设备上实现大模型的部署而无需再训练。
• 在语言模型（OPT、Qwen、LLaMA）和视觉语言模型（CogVLM2）上演示该方法，显示保留的性能。

提出的方法

• 采用基于分块的多协同闭环补偿，使用残差来缓解分块间误差累积。
• 使用两阶段量化：阶段1 依据 Hessian 信息进行本地优化，获得初始分块量化。
• 阶段2 进行多轮、类似 Gauss-Seidel 的残差驱动更新，结合内存中的全局 Hessian 进行分块细化。
• 引入单实例标定范式，预先计算的全局 Hessian 保留，仅在细化阶段使用最后一个标定批次。
• 提供带步长 alpha 的线性更新方案，以稳定分块更新。
• 利用瞬时 Hessian 曲率重建来引导每块的量化，而无需重新加载校准数据。

Figure 1 : Block based multi-collaborative closed-loop compensation.

实验结果

研究问题

• RQ1 与传统的一次性分块量化相比，基于残差驱动的多协同补偿是否能减少分块间量化误差累积？
• RQ2 基于瞬时 Hessian 曲率的单实例标定是否能在避免再次加载完整校准数据的同时保留全局二阶信息？
• RQ3 RPIQ 在将大语言模型和视觉语言模型压缩到 4 位表示时，在维持对视觉障碍辅助相关任务性能方面表现如何？
• RQ4 在资源受限的辅助设备上，该方法带来哪些内存和运行时方面的收益？

主要发现

• RPIQ 实现了约 60-75% 峰值内存减量的 4 位量化，相对于全精度模型。
• 该方法在多项语言与视觉任务中保持的性能接近全精度模型。
• 分块级残差协作有效缓解了大模型中的分块间误差累积。
• 单实例标定在不反复加载校准数据的前提下保留全局二阶信息，提升效率。
• Gauss-Seidel 式迭代量化为辅助场景下的大模型提供了鲁棒且更快的收敛性。

Figure 2 : Single instance calibration paradigm.