SAYO：用强化学习强化多模态大语言模型的视觉注意力
Do MLLMs Really See It: Reinforcing Visual Attention in Multimodal LLMs
AI 导读
研究提出 SAYO，一种通过强化学习训练的视觉推理模型，引入区域级视觉注意力奖励，将优化信号与视觉依据的推理步骤对齐。分析显示现有 MLLM 视觉聚焦弱，早期视觉错位在后续推理中很少被纠正，导致错误传播。多项多模态基准实验表明 SAYO 在推理与感知任务上持续提升性能，项目已开源。
正文 · 原文
Abstract:While chain-of-thought (CoT) reasoning has substantially improved multimodal large language models (MLLMs) on complex reasoning tasks, existing approaches largely rely on long textual reasoning trajectories and provide limited mechanisms for learning stable visual attention policies. Our analysis shows that current MLLMs exhibit weak visual focus: early-stage visual misalignment is rarely corrected during subsequent reasoning, leading to error propagation and failed inferences. We argue that this limitation stems from inadequate credit assignment for visual attention during training. To address this issue, we propose SAYO, a visual reasoning model trained with a reinforcement learning (RL) framework that introduces a region-level visual attention-based reward. This reward explicitly aligns optimization signals with visually grounded reasoning steps, enabling the model to learn more reliable attention behaviors. Extensive experiments across multiple multimodal benchmarks demonstrate that SAYO consistently improves performance on diverse reasoning and perception tasks. Project is open at this https URL.
| Subjects: | Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV) |
| Cite as: | arXiv:2602.08241 [cs.AI] |
| (or arXiv:2602.08241v2 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2602.08241 arXiv-issued DOI via DataCite |
Submission history
From: Siqu Ou [view email]
[v1]
Mon, 9 Feb 2026 03:33:23 UTC (2,546 KB)
[v2]
Sun, 25 Oct 2026 09:51:06 UTC (2,530 KB)
来源：arXiv cs.AI · arxiv.org