arXiv cs.AI· Paul J\"unger, Justin Lovelace, Linxi Zhao, Dongyoung Go, Kilian Q. Weinberger·· 5 小时前AI 评分52
SARDI：面向扩散语言模型的免训练自增强检索框架
Self-Augmenting Retrieval for Diffusion Language Models
AI 导读
论文提出 SARDI，一种面向扩散语言模型的动态 RAG 框架，利用去噪过程中的投机性 token 预测引导检索，无需训练、不依赖特定检索器，适用于任何具备推理能力的离散扩散语言模型。
正文 · 原文
Abstract:Diffusion language models generate text by iteratively denoising an entire response in parallel, holding a noisy draft of the answer at every step. We show that these intermediate states surface salient entities early in the denoising trajectory, making them a useful lookahead signal for retrieval-augmented generation (RAG). We further demonstrate that grounding generation in retrieved evidence sharply reduces inter-token dependence, making RAG particularly well suited to parallel decoding. We exploit both through Self-Augmenting Retrieval for Diffusion Language Models (SARDI), a dynamic RAG framework that uses speculative token predictions to guide retrieval during denoising. SARDI is training-free, retriever-agnostic, and applicable to any reasoning-capable discrete diffusion language model. Across five multi-hop QA benchmarks, SARDI outperforms training-free diffusion and autoregressive retrieval baselines at up to 4x higher throughput.
| Comments: | ICML 2026 |
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Machine Learning (cs.LG) |
| Cite as: | arXiv:2606.06474 [cs.CL] |
| (or arXiv:2606.06474v2 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2606.06474 arXiv-issued DOI via DataCite |
Submission history
From: Paul Jünger [view email]
[v1]
Thu, 4 Jun 2026 17:56:27 UTC (202 KB)
[v2]
Thu, 29 Oct 2026 16:45:03 UTC (262 KB)
来源：arXiv cs.AI · arxiv.org