arXiv cs.AI· Jaeyong Song, Daniel Wang, Pratyush Patel, Jonny Kong, Preyas Shah, Shobhit Kanaujia·· 3 小时前AI 评分33
ForeSpark：面向并行推测解码的草稿与验证预算联合调度
ForeSpark: Joint Draft and Verification Budget Scheduling for Parallel Speculative Decoding
AI 导读
论文提出工作保持推测（WCS）设计并实现原型系统 ForeSpark，在 LLM 服务中为每个请求联合分配草稿与验证预算。系统包含上下文感知的预草稿估计器、基于全局排序的轻量级联合调度器和草稿后预算分配。在 vLLM 上实现后，ForeSpark 吞吐较 DSpark 最高提升 1.77x，较自回归解码提升 2.49x，且在延迟 SLO 下可承受最高 1.51x 于 DSpark 的请求率。
正文 · 原文
Abstract:Parallel drafters accelerate LLM decoding by proposing a whole block of draft tokens in a single forward pass, making them attractive for LLM serving. Recent systems optimize goodput (i.e., accepted tokens per second) because rejected drafts steal compute from other requests. However, these systems control only how many tokens are verified, so every request still drafts a full block. At high serving load, we find that drafting consumes up to 40% of step time, of which a majority is discarded because it is never verified. Yet, draft-length control from autoregressive drafters does not carry over, because a parallel drafter commits to its length before its single pass. Moreover, draft and verification lengths are coupled across the batch, so they must be selected jointly for every request while keeping the decision computationally cheap at every step. In this paper, we propose work-conserving speculation (WCS), a design that jointly allocates draft and verification budgets per request in LLM serving, and present ForeSpark as its prototype. ForeSpark realizes WCS with (i) a context-aware pre-draft estimator that predicts acceptance for every candidate length from the target model's context features, (ii) a lightweight joint draft-verify scheduler that selects both lengths through a global sort, and (iii) a post-draft budget distribution that spends verification on the most promising tokens. Implemented on vLLM, ForeSpark improves throughput by up to 1.77x over the state-of-the-art DSpark and by 2.49x over autoregressive decoding at a matched per-request generation rate, without changing the output distribution. It also tolerates higher load, sustaining up to 1.51x the request rate of DSpark under latency service level objectives (SLOs).
| Comments: | 21 pages |
| Subjects: | Distributed, Parallel, and Cluster Computing (cs.DC); Artificial Intelligence (cs.AI); Machine Learning (cs.LG) |
| Cite as: | arXiv:2611.13442 [cs.DC] |
| (or arXiv:2611.13442v1 [cs.DC] for this version) | |
| https://doi.org/10.48550/arXiv.2611.13442 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Jaeyong Song [view email]
[v1]
Tue, 10 Nov 2026 01:37:38 UTC (837 KB)
来源：arXiv cs.AI · arxiv.org