双向中心化得分匹配（BCSM）用于离散扩散蒸馏
Bidirectional Centered Score Matching for Discrete Diffusion Distillation
AI 导读
一篇 arXiv 论文提出双向中心化得分匹配（BCSM），一种高效的离散扩散模型位置内蒸馏目标，将成对比较从 O(|V|²) 复杂度降至 O(K)，并引入跨时间步转移对齐。实验显示单步推理下 NFE 最高减少 6 倍，HumanEval 相对提升 16.9%，MMMU 相对提升 15.1%。
正文 · 原文
Abstract:Discrete diffusion models with parallel decoding are often distilled using token-wise KL objectives. Although token-wise KL exactly matches the corresponding product-form block distributions, its probability-space optimization geometry and teacher-only candidate truncation may provide insufficient supervision under aggressive few-step distillation. Concrete Score Matching (CSM) provides an alternative within-position objective by aligning relative logit geometry, but its complete pairwise form incurs \mathcal{O}(|\mathcal{V}|^2) complexity. We propose Bidirectional Centered Score Matching (BCSM), an efficient within-position distillation objective that constructs an \mathcal{O}(K) bidirectional active set, reduces pairwise comparisons to \mathcal{O}(K^2), and factorizes them into an \mathcal{O}(K) objective. We further introduce cross-timestep transition alignment. Experiments demonstrate strong quality--efficiency trade-offs, including up to 6\times NFE reduction with a 16.9\% relative HumanEval gain and a 15.1\% relative MMMU gain at single-step inference.
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2611.04279 [cs.AI] |
| (or arXiv:2611.04279v2 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2611.04279 arXiv-issued DOI via DataCite |
Submission history
From: Yh.Peng Tu [view email]
[v1]
Tue, 3 Nov 2026 04:05:41 UTC (7,225 KB)
[v2]
Wed, 4 Nov 2026 03:21:50 UTC (7,225 KB)
来源：arXiv cs.AI · arxiv.org