arXiv cs.AI· Zhuoming Liu, Jinhong Lin, Kwan Man Cheng, Lin Zhang, Shayok Bagchi, Yin Li·· 4 小时前AI 评分38
DRIFT：一种用于视觉语言模型连续输出解码的残差流匹配适配器
DRIFT: A Residual Flow Adapter for Decoding Continuous Outputs in Vision-Language Models
AI 导读
研究者提出 DRIFT 框架，将预训练视觉语言模型（VLM）适配到连续输出解码任务。该方法由基础预测器提供粗估计，再用基于 flow matching 的生成式精炼模块迭代改进，把全局分布建模简化为围绕强先验的局部残差分布。在视觉定位和机器人控制等感知与规划任务上，DRIFT 在 MLLM、VLA、WAM 多种架构中持续优于回归和生成式基线。
正文 · 原文
Abstract:Many modern vision-language models (VLMs) build on autoregressive decoding of discrete tokens. While text-based output interfaces enable scalable pretraining and strong zero-shot generalization across diverse tasks, they are poorly suited for problems that require precise continuous outputs, such as localizing temporal boundaries of events or generating robotic control actions. To address this challenge, we propose DRIFT, a general framework for adapting pretrained VLMs to continuous decoding tasks. DRIFT combines a base predictor, which provides a coarse estimate of the target output, with a generative refinement module based on flow matching that iteratively improves the prediction. This residual formulation transforms the generative modeling problem from learning a global output distribution to modeling a localized residual distribution around a strong prior, substantially simplifying optimization. We evaluate DRIFT on both perception and planning tasks, including visual grounding and robotic control. Across multiple tasks and architectures spanning MLLMs, VLAs, and WAMs, DRIFT consistently outperforms a strong set of regression- and generative-based solutions.
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Machine Learning (cs.LG) |
| Cite as: | arXiv:2606.05758 [cs.CV] |
| (or arXiv:2606.05758v2 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2606.05758 arXiv-issued DOI via DataCite |
Submission history
From: Zhuoming Liu [view email]
[v1]
Thu, 4 Jun 2026 06:37:10 UTC (3,544 KB)
[v2]
Tue, 27 Oct 2026 15:50:15 UTC (3,547 KB)
来源：arXiv cs.AI · arxiv.org