arXiv cs.AI· Zhongxing Xu, Zhonghua Wang, Zhe Qian, Shiyan Su, Ming Hu, Xiaocheng Zou, Wei Feng, Mingquan Lin, Yuyin Zhou, Yifan Peng, Hamid Rezatofighi, Zongyuan Ge·· 3 小时前AI 评分28
StepRefine：用步数感知解码缓解多模态扩散语言模型幻觉
Rethinking in Spikes: Mitigating Hallucinations in MDLMs with Step-Aware Decoding
AI 导读
研究者提出 StepRefine，一种即插即用的解码策略，用于缓解多模态扩散语言模型（MDLM）的幻觉。该方法发现掩码不确定性在迭代去噪中整体下降，少数去噪步出现与幻觉相关的局部熵尖峰，于是通过步数感知的切换，在熵尖峰步采用概率加权嵌入并多轮连续精炼，由精炼控制器监控分布稳定性以减少多余循环。实验显示 StepRefine 在多模态基准上显著降低不同 MDLM 的幻觉。
正文 · 原文
Authors:Zhongxing Xu, Zhonghua Wang, Zhe Qian, Shiyan Su, Ming Hu, Xiaocheng Zou, Wei Feng, Mingquan Lin, Yuyin Zhou, Yifan Peng, Hamid Rezatofighi, Zongyuan Ge
Abstract:Recent advancements in multimodal diffusion language models (MDLMs) have exhibited strong global modeling capabilities through iterative refinement. We observe that mask uncertainty follows an overall downward trend during iterative denoising, while a few denoising steps exhibit localized entropy spikes closely associated with hallucinations. We argue that anomalous steps can be identified by measuring the entropy deviation of each denoising step from its neighboring temporal window. We hypothesize that discrete token commitment discards the distributional information at masked positions, limiting the continued interaction between candidate semantics and the context during entropy spikes. With this goal, we present StepRefine, a plug-and-play decoding strategy that leverages semantic context to achieve reliable denoising. The core of our method lies in a step-aware switch between discrete denoising and latent refinement. The model employs probability-weighted embeddings at entropy spike steps to preserve diverse reasoning hypotheses and performs multiple rounds of continuous refinement. Meanwhile, a refinement controller monitors distributional stability to reduce unnecessary loops and switch the model back to standard discrete denoising. Through extensive experiments, StepRefine demonstrates significant hallucination mitigation across different MDLMs on multimodal benchmarks.
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2611.12901 [cs.CL] |
| (or arXiv:2611.12901v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2611.12901 arXiv-issued DOI via DataCite |
Submission history
From: Zhonghua Wang [view email]
[v1]
Wed, 14 Oct 2026 07:52:29 UTC (4,035 KB)
来源：arXiv cs.AI · arxiv.org