arXiv cs.AI· Hao-Yuan He, Peng-Fei Liu, Chen Chen, Si Shen, Quan He, Runzhou Liu, Xinqi Li, Zhenhua Dong, Ming Li·· 6 小时前AI 评分39
LongSpark：用固定成本并行起草器实现高效投机解码
LongSpark: Efficient speculative decoding with a fixed-cost parallel drafter
AI 导读
arXiv 论文提出 LongSpark，一种基于块扩散（block-diffusion）的投机解码起草器，通过从目标模型验证过程中提取固定尺寸、多尺度视图，使起草器解码成本与上下文长度完全无关。在多模型规模和真实服务条件下，LongSpark 取得 SOTA 端到端效率，在长上下文任务上实现最低的每输出 token 耗时，同时将起草器上下文状态缩减数个数量级。
正文 · 原文
Abstract:Speculative decoding accelerates autoregressive inference by verifying multiple draft tokens in a single target forward pass. However, as the context grows, existing state-of-the-art drafters become increasingly expensive, eroding the very efficiency advantage they are designed to provide. We argue that this scaling is unnecessary. A standalone language model must grow with its prefix because it is solely responsible for every token it produces. A drafter, by contrast, only proposes candidates; the target catches and corrects every error before any token is committed. The drafter's decoding cost can therefore be made entirely independent of the prefix length. We introduce LongSpark, a block-diffusion drafter that achieves this by extracting fixed-size, multiscale views from the target's verification pass, thereby eliminating the need for a growing persistent state. Extensive evaluations demonstrate that LongSpark achieves state-of-the-art end-to-end efficiency across multiple model scales and realistic serving conditions. Notably, it delivers the lowest time-per-output-token on long-context tasks while reducing the drafter's context state by several orders of magnitude.
| Comments: | update authorships & comparsions |
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2609.37029 [cs.LG] |
| (or arXiv:2609.37029v2 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2609.37029 arXiv-issued DOI via DataCite |
Submission history
From: Hao-Yuan He [view email]
[v1]
Tue, 29 Sep 2026 08:42:58 UTC (372 KB)
[v2]
Mon, 26 Oct 2026 09:15:06 UTC (373 KB)
来源：arXiv cs.AI · arxiv.org