arXiv cs.AI· Leon Sick, Lukas Hoyer, Yannick Str\"umpler, Christian Reisswig, Timo Ropinski, Federico Tombari·· 3 小时前AI 评分40
On-Policy Self-Training：大语言模型的无数据自我训练方法
On-Policy Self-Training: Data-Free Self-Improvement in Large Language Models
AI 导读
一篇 arXiv 论文提出 On-Policy Self-Training 框架，将 on-policy 自蒸馏重构为无需外部监督的自我训练方法：模型仅凭粗粒度主题描述，自生成问答对与特权推理演示作为课程。该框架在文本基准和视觉问答（VQA）上实现持续自我提升，且不损害未被训练领域的原有能力，表明基础模型蕴含可被放大的潜在能力。
正文 · 原文
Abstract:On-policy self-distillation enables post-training foundation models without relying on stronger external teachers. However, existing approaches rely on ground-truth question-answer pairs, human-curated problem sets, or other supervised signals. In this work, we reformulate on-policy self-distillation as a self-training framework, driven by self-generated question-answer pairs and privileged reasoning demonstrations. We push supervision to the limit: guided solely by coarse, high-level topic descriptions, the model synthesizes its own curriculum of queries and privileged reasoning traces without access to any human-curated prompts or answers. We find that self-distillation on fully synthetic privileged information enables a model to improve across multiple domains while retaining its capabilities on domains it is not self-trained on. We demonstrate the generality of this framework across text and multimodal domains, achieving consistent self-improvement on standard language benchmarks and Visual Question Answering. Our findings demonstrate that foundation models possess rich latent capabilities that can be amplified without external task supervision or curated problem distributions.
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Machine Learning (cs.LG) |
| Cite as: | arXiv:2610.32770 [cs.CL] |
| (or arXiv:2610.32770v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.32770 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Leon Sick [view email]
[v1]
Fri, 23 Oct 2026 15:12:03 UTC (152 KB)
来源：arXiv cs.AI · arxiv.org