arXiv cs.AI· Giannis Daras, Marios Papachristou, Adam Klivans, Antonio Torralba, Constantinos Daskalakis·· 2 小时前AI 评分38
Ambient LLMs:通过温度匹配实现数据高效的监督微调
Ambient LLMs: Data-Efficient Supervised Fine-tuning via Temperature Matching
AI 导读
arXiv 论文提出 Ambient SFT 框架,用冻结的预训练模型作为噪声教师,要求微调后模型在高温条件分布上与预训练模型匹配,以缓解 SFT 数据集偏小导致的过拟合与尾部灾难性遗忘。温度超参数控制对预训练模型的信任程度:T→∞ 退回标准 SFT,T=1 等同知识蒸馏。该方法在领域适配、指令微调和推理任务上均优于标准 SFT,平均相对提升最高达 21%。
正文 · 原文
Abstract:Collecting large-scale, diverse, and high-quality post-training datasets for autoregressive language models is challenging. Consequently, Supervised Fine-Tuning (SFT) is typically applied to datasets significantly smaller than those used for pre-training, which leads to overfitting and the catastrophic forgetting of the tails of the data distribution. To prevent this collapse, we introduce Ambient SFT, a principled framework for leveraging imperfect predictions to augment the fine-tuning set. In our method, a "frozen" version of the pre-trained model serves as a noisy teacher that the fine-tuned model must match at a coarse level. In particular, we require that the high-temperature versions of the conditional distributions produced by the post-training model should match the high-temperature versions of the pre-training ones. The temperature hyperparameter controls the level of trust in the pre-trained model, with the limit $T\to \infty$ reverting back to regular SFT and $T=1$ to knowledge distillation. We demonstrate that our approach outperforms standard SFT across domain adaptation, instruction-tuning, and reasoning, leading for up to $21\%$ average relative improvement to standard SFT.
| Comments: | Work in progress |
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computation and Language (cs.CL) |
| Cite as: | arXiv:2611.23229 [cs.LG] |
| (or arXiv:2611.23229v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2611.23229 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Giannis Daras [view email]
[v1]
Tue, 17 Nov 2026 18:59:37 UTC (171 KB)
来源:arXiv cs.AI · arxiv.org