掩码扩散与随机序吸附：一个可精确求解的并行解码模型
Masked diffusion as random sequential adsorption: an exactly solvable model for parallel decoding
AI 导读
一篇 arXiv 论文将掩码扩散模型的并行解码误差映射到驱动多粒子系统的统计物理。论文证明并行生成的误差等于生成过程相对反向掩码过程的熵产生，并对 Markov 数据给出带标记间隙的随机序吸附过程的精确解。最优解掩码调度是该热力学度量的测地线，小步起步的调度在困惑度和路径误差上收益更大且所需步数更少。
正文 · 原文
Abstract:Masked diffusion models generate text by reversing a stochastic masking process. Unlike autoregressive LLMs, which generate text one token at a time, they can reveal many tokens in parallel, at the price of an error, since tokens revealed together are drawn independently. We show that this trade-off maps onto the statistical physics of driven many-particle systems. (i) For any data, the error of parallel generation is the entropy production of the generation process relative to the reversed masking process. (ii) For Markov data, generation is a random sequential adsorption process with labelled gaps, which we solve exactly for any number of steps. (iii) For many steps, the error becomes the dissipation of a slowly driven system: the unmasking schedule plays the role of the driving protocol, the number of steps that of its duration, and the error is set by a thermodynamic metric that we obtain in closed form from the mutual information between tokens. Optimal schedules are geodesics of this metric. The exact solution shows how correlations in the text shape both the error and the quality of the output. The longer the correlations, the more the error concentrates at the start of generation. Schedules that start with small steps then gain more, both in path error and in perplexity, and need fewer steps to reach their asymptotic behaviour.
| Comments: | 14 pages (incl. appendices), 3 figures |
| Subjects: | Statistical Mechanics (cond-mat.stat-mech); Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Machine Learning (cs.LG) |
| Cite as: | arXiv:2611.07845 [cond-mat.stat-mech] |
| (or arXiv:2611.07845v1 [cond-mat.stat-mech] for this version) | |
| https://doi.org/10.48550/arXiv.2611.07845 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Adrian Baule [view email]
[v1]
Thu, 5 Nov 2026 11:23:40 UTC (50 KB)
来源：arXiv cs.AI · arxiv.org