arXiv cs.AI· Luxi Lin, Zhihang Lin, Zhanpeng Zeng, Yuhao Chen, Qingyu Zhang, Jixiang Luo, Xuelong Li, Rongrong Ji·· 4 小时前AI 评分35
EDA:参数与数据高效的投机解码草稿模型对齐框架
Efficiently Aligning Draft Models via Parameter- and Data-Efficient Adaptation
AI 导读
arXiv 论文提出高效草稿模型适配框架 EDA,解决目标模型经领域微调后投机解码性能下降的问题,避免为每个目标模型重训草稿模型的高成本。方法包含三项创新:仅更新轻量私有组件的解耦架构、利用微调后目标模型再生训练数据的策略,以及优先高价值数据的样本选择机制。实验显示 EDA 能在微调模型上恢复投机解码性能,平均接受长度优于全量重训,且训练成本显著降低。
正文 · 原文
Abstract:Speculative decoding accelerates LLM inference but suffers from performance degradation when target models are fine-tuned for specific domains. A naive solution is to retrain draft models for every target model, which is costly and inefficient. To address this, we introduce a practical parameter- and data-efficient framework named Efficient Draft Adaptation, abbreviated as EDA, for efficiently adapting draft models. EDA introduces three innovations: (1) a decoupled architecture that utilizes shared and private components to model the shared and target-specific output distributions separately, enabling parameter-efficient adaptation by updating only the lightweight private component; (2) a data regeneration strategy that utilizes the fine-tuned target model to regenerate training data, thereby improving the alignment between training and speculative decoding, leading to higher average acceptance length; (3) a sample selection mechanism that prioritizes high-value data for efficient adaptation. Our experiments show that EDA effectively restores speculative performance on fine-tuned models, achieving superior average acceptance lengths with significantly reduced training costs compared to full retraining. Code is available at this https URL.
| Comments: | 10 pages |
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2603.09527 [cs.LG] |
| (or arXiv:2603.09527v2 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2603.09527 arXiv-issued DOI via DataCite |
Submission history
From: Luxi Lin [view email]
[v1]
Tue, 10 Mar 2026 11:35:58 UTC (1,339 KB)
[v2]
Fri, 13 Nov 2026 09:41:09 UTC (1,356 KB)
来源:arXiv cs.AI · arxiv.org