扩散语言模型并行解码的耦合度测量方法
Measured Coupling for Parallel Decoding in Diffusion Language Models
AI 导读
一项 arXiv 研究提出用耦合度测量改进扩散语言模型的并行解码：通过主前向传播中的低成本探针，测量提交一个 token 对其他位置模型置信度的影响，替代均场解码的近似方法，无需训练或额外模型。在三个模型和四个基准上，该方法所需的步数远少于均场解码，多数设置下（包括使用 key-value cache 时）速度更快，平均精度损失仅 1.1 点；打乱或替换耦合会失去该增益。
正文 · 原文
Abstract:Diffusion language models can write many tokens per step, but must choose which tokens are safe to commit together. Mean-field decoding penalizes pairs of positions whose predictions look alike. We replace this proxy with the coupling itself: how much committing one token changes the model's confidence at another. Cheap probes inside the main forward pass measure it, with no training or extra model, and it plugs into the same mean-field rule. On three models and four benchmarks, our method needs far fewer steps than mean-field decoding and is faster in most settings, also with a key-value cache, at an average accuracy cost of 1.1 points. Shuffling the couplings or replacing them with a uniform bonus loses this gain.
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Distributed, Parallel, and Cluster Computing (cs.DC); Machine Learning (cs.LG); Performance (cs.PF) |
| Cite as: | arXiv:2611.08307 [cs.CL] |
| (or arXiv:2611.08307v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2611.08307 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Tal Oved [view email]
[v1]
Thu, 5 Nov 2026 17:10:03 UTC (257 KB)
来源：arXiv cs.AI · arxiv.org