arXiv cs.AI· Sinuo Wang, Jiacheng Lu, Zeyu Zhang, Maojia Song, Rui Sun, Beidi Luan, Yutong Xie, Jing Li, Daxin Jiang, Qi Wu, Zuo Bai·· 4 小时前AI 评分33
P$^2$-MOPD：利用参数先验加速多教师在线策略蒸馏
P$^2$-MOPD: Accelerating Multi-Teacher On-Policy Distillation via Parameter Priors
AI 导读
P$^2$-MOPD 将学生与专家参数的线性组合视为单个可微模型，在相同的在线策略蒸馏目标下联合优化混合系数与学生权重，使专家充当参数先验。在 1.7B 和 4B 规模的四个领域中，其收敛速度更快且更均衡，仅用一半训练步数即超过 MOPD 的最终平均恢复率。训练结束时，1.7B 模型平均恢复率达 94.7%，而 MOPD 为 80.9%。
正文 · 原文
Authors:Sinuo Wang, Jiacheng Lu, Zeyu Zhang, Maojia Song, Rui Sun, Beidi Luan, Yutong Xie, Jing Li, Daxin Jiang, Qi Wu, Zuo Bai
Abstract:Multi-teacher on-policy distillation consolidates expert capabilities by supervising student-generated trajectories, whereas task arithmetic offers a training-free alternative through linear parameter combinations. We view the linear combination of student and expert parameters as a single differentiable model and propose P$^2$-MOPD, which jointly optimizes the mixing coefficients and the student weights under the same on-policy distillation objective. Unlike task arithmetic, which stays within the span of the frozen student and expert parameters for any choice of merging coefficients, P$^2$-MOPD keeps the student trainable, so the experts act as parameter priors without confining it to this span. Across four domains at 1.7B and 4B scales, P$^2$-MOPD achieves faster and more balanced recovery than MOPD from the same initialization, surpassing its final average recovery in half the training steps. At the end of training, our 1.7B model reaches 94.7% average recovery, versus 80.9% for MOPD.
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2611.03953 [cs.LG] |
| (or arXiv:2611.03953v2 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2611.03953 arXiv-issued DOI via DataCite |
Submission history
From: Sinuo Wang [view email]
[v1]
Mon, 2 Nov 2026 20:53:15 UTC (280 KB)
[v2]
Thu, 5 Nov 2026 07:14:40 UTC (280 KB)
来源：arXiv cs.AI · arxiv.org