DForkTree：用分支感知的块扩散草稿树加速投机解码
DForkTree: Accelerating Speculative Decoding with Fork-Aware Block Diffusion Draft Trees
AI 导读
DForkTree 是一种免训练的投机解码加速方法，为草稿树的每个分支在其自身 token 上重新锚定，并用目标模型生成 n-gram 的点互信息调整草打分，通过逐层 GPU 搜索恢复最优树。在 Qwen3.5-4B 和 9B 上，DForkTree 全面超越基线，较 DFlash 最高提升 41.5%、较 DDTree 提升 15.1%，相对自回归解码平均加速最高达 4.75x。
正文 · 原文
Abstract:Speculative decoding speeds up language models by having a lightweight drafter propose future tokens that the target model verifies in parallel. Block diffusion drafters such as DFlash predict an entire block in one forward pass, and DDTree builds draft trees directly from their per-position distributions. Each distribution implicitly assumes that the drafter's own greedy continuation is accepted, whereas a draft tree improves over a chain only on branches that leave this greedy path. Once a path forks, the drafter's top-1 accuracy drops from 89% to 18% and its top candidates often miss the true token. Re-running the drafter from the fork token largely restores its accuracy, showing that the errors stem from conditioning on an outdated prefix rather than from limited model capacity. In this paper, we introduce DForkTree, a training-free method that re-anchors each branch on its own tokens. It adjusts the drafter's scores with pointwise mutual information from target-generated n-grams, weighted more heavily after a fork, and a layer-wise GPU search recovers the exact optimal tree under these scores. On Qwen3.5-4B and 9B, DForkTree outperforms all baselines on every benchmark, by up to 41.5% over DFlash and 15.1% over DDTree, with average speedups of up to 4.75x over autoregressive decoding.
| Comments: | 21 pages, 12 figures, 13 tables. Code: this https URL |
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2611.12356 [cs.AI] |
| (or arXiv:2611.12356v2 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2611.12356 arXiv-issued DOI via DataCite |
Submission history
From: Zhicheng Zheng [view email]
[v1]
Mon, 9 Nov 2026 13:06:52 UTC (471 KB)
[v2]
Thu, 12 Nov 2026 06:04:02 UTC (474 KB)
来源：arXiv cs.AI · arxiv.org