arXiv cs.AI· Xinsong Feng, Chenan Wang, Leshu Tang, Johnny Liu, Dong Li, Da Li, Haipeng Chen·· 2 小时前AI 评分34
DELA：为扩散语言模型学习可控编辑的插件框架
DELA: Learning Controllable Edits for Diffusion Language Models
AI 导读
arXiv 论文提出 DELA（Diffusion Editing Language Adapter），一个让冻结的扩散语言模型实现可控草稿精修的插件框架。DELA 通过 Insert、Delete、Remask、Stop 四种显式编辑动作分离高层编辑控制与 token 级生成，并采用两阶段解码：支持 KV-cache 复用的快速追加式起草加全局精修。
正文 · 原文
Abstract:Diffusion language models (DLMs) support bidirectional denoising and iterative refinement, but generation remains largely controlled by token-level masking schedules. This leaves draft-level refinement implicit and limits explicit control over when to insert missing content, delete redundant spans, revisit uncertain regions, or stop editing. We introduce Diffusion Editing Language Adapter (DELA), a plug-in framework that enables controllable draft refinement with a frozen DLM. DELA operates on clean, variable-length drafts using four explicit editing actions: Insert, Delete, Remask, and Stop. The controller determines when and where to edit the draft, while the frozen DLM fills masked regions through denoising, separating high-level edit control from token-level generation. A naive single-phase editor would be computationally expensive because arbitrary interior edits can disrupt prefix stability and limit reliable KV-cache reuse. To address this, DELA uses a two-phase decoding procedure: a fast, append-only drafting phase that supports KV-cache reuse, followed by a global refinement phase that enables targeted non-monotonic edits to the full draft. We train the controller using a staged curriculum that begins with learning local edit priors from synthetic recovery tasks and mined actions that improve draft quality. Subsequent stages refine edit selection through rollout-based outcome preferences and alignment with downstream task feedback. Experiments on mathematical reasoning and code generation show that DELA achieves gains of up to 5.5 percentage points over direct diffusion decoding while keeping the backbone frozen. DELA thus provides a modular approach to explicit draft-level control in diffusion language models.
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.32096 [cs.AI] |
| (or arXiv:2610.32096v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.32096 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Xinsong Feng [view email]
[v1]
Fri, 23 Oct 2026 02:57:27 UTC (228 KB)
来源：arXiv cs.AI · arxiv.org