RECAP：面向多模态投机解码的请求本地与持久记忆预测复用方法
RECAP: Context-Aware Prediction Recycling with Request-Local and Persistent Memories for Multimodal Speculative Decoding
AI 导读
论文提出 RECAP，一种免训练的多模态投机解码预测复用方法，通过一至三 token 键存储目标模型预测，区分请求内上下文记忆与跨请求持久记忆，并以熵置信度加权融合检索结果。
正文 · 原文
Abstract:Recycling a target model's own predictions offers a training-free source of proposals for speculative decoding. In vision-language models, however, existing recycling indexes predictions by the last token alone, conflating predictions computed under different images and prefixes; we find that such reuse fails most often at image-sensitive decoding states. Longer keys reduce this aliasing but match fewer states. We introduce \method, which stores target predictions under one-, two-, and three-token keys: contextual memories live only for the current request and are thus implicitly conditioned on its image, while token-level memory persists across requests to retain coverage. At each lookup, entropy-confidence weighting fuses the retrieved predictions according to their key specificity and concentration. Best-first search composes the fused transitions into a candidate tree, supplemented by suffix-matched continuations, and one target pass verifies the tree under greedy decoding. Without training or an auxiliary draft model, \method\ achieves mean speedups over autoregressive decoding of $1.66\speed$ and $1.56\speed$ across seven single-turn benchmarks on Qwen2.5-VL-7B and LLaVA-1.6-7B, exceeding the strongest training-free baseline, Token Recycling ($1.51\speed$ and $1.37\speed$). Our implementation is available in this https URL.
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2611.05894 [cs.AI] |
| (or arXiv:2611.05894v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2611.05894 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Hengjie Zhu [view email]
[v1]
Wed, 4 Nov 2026 03:32:18 UTC (858 KB)
来源：arXiv cs.AI · arxiv.org