Cord：面向掩码扩散语言模型的置信度锚定残差解码方法
Cord: Confidence-Anchored Order Residual Decoding for Masked Diffusion Language Models
AI 导读
研究者提出 Cord，一种用于掩码扩散语言模型（dLLM）解码顺序的轻量级 prompt 条件策略，以置信度解码为参考，通过几何残差学习何时偏离该参考。在每个 rollout 中仅采样一个低维系数矩阵即可诱导完整解码轨迹，以 contextual-bandit 方式直接从序列级奖励训练。
正文 · 原文
Abstract:The decoding order of masked diffusion language models (dLLMs) can substantially affect generation quality, yet common decoders typically apply the same reveal rule throughout generation. Confidence provides a strong default, but it measures how decisively a position can be predicted now, not how useful revealing that position will be for subsequent decoding. We propose Cord, a lightweight prompt-conditioned policy that retains confidence decoding as a stable reference and learns when and how to depart from it through geometric residuals over generation direction, proximity to revealed context, and two-sided infilling. The residuals adapt to both the prompt and the evolving partial response. Rather than sampling a new policy action at every decoding step, Cord samples a single low-dimensional coefficient matrix per rollout, which induces an entire state-dependent reveal trajectory. This yields a contextual-bandit formulation trained directly from sequence-level rewards, without step-level credit assignment or backbone updates. Across three dLLMs and five reasoning, code, and constraint-satisfaction benchmarks, Cord improves average task success from 32.0% under confidence decoding to 48.2%, and further improves over the best per-domain fixed residual. Behavioral audits show that the learned residuals induce substantial, task-dependent changes in reveal locality and two-sided infilling. Cord adds only $0.53$M trainable parameters, requires no additional backbone forward passes, and incurs only 5.39% inference latency overhead.
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.39372 [cs.CL] |
| (or arXiv:2610.39372v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.39372 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Chengwei Zhou [view email]
[v1]
Thu, 29 Oct 2026 05:03:29 UTC (777 KB)
来源：arXiv cs.AI · arxiv.org