TTCD：面向语言建模的逐 Token 连续时间扩散模型
Token Time Continuous Diffusion for Language Modeling
AI 导读
研究者提出 token time continuous diffusion（TTCD），一种在连续空间工作的扩散语言模型，将高斯噪声确定性映射为最终 token 序列，并引入逐 token 时间概念，使更确定的 token 更快生成。
正文 · 原文
Abstract:In this paper we introduce token time continuous diffusion (TTCD), a new diffusion language model which (a) operates in continuous space, deterministically mapping Gaussian noise to a final token canvas with no further sampling, and crucially (b) incorporates a new notion of per-token times, with some tokens proceeding from noise to token at a faster rate than others. Continuous space modeling helps TTCD avoid the parallel sampling of multiple tokens, which is a key source of inaccuracy at high speedups for models that iterate purely in discrete space. The notion of per-token times helps TTCD to better model conditional generation, allows for more sure tokens to proceed at a faster rate, and allows for differentiated inter-token influences during refinement. TTCD outperforms discrete models at high speedups. We train a 160M parameter TTCD model on OpenWebText, and then self-distill it; we find that at high speedups we are comparable in unconditional generation quality, and outperform in conditional generation, several existing models of similar size trained, on the same data, and self-distilled. We achieve similar gains in Sudoku solving as well.
| Comments: | Accepted to NeurIPS 2026 |
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2607.14106 [cs.CL] |
| (or arXiv:2607.14106v2 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2607.14106 arXiv-issued DOI via DataCite |
Submission history
From: Parikshit Bansal [view email]
[v1]
Thu, 7 May 2026 16:23:06 UTC (659 KB)
[v2]
Wed, 4 Nov 2026 16:32:30 UTC (1,121 KB)
来源：arXiv cs.AI · arxiv.org