跳到正文
arXiv cs.AI· Lirui Luo, Kelong Mao, Xinwei Yang, Rongqing Li, Luyu Chen, Kieran Wong, Yudong Guo, Xinrui Wang, Jiayin Zhu, Gu Simiu, Sulong Xu, Cong Fang·· 1 小时前AI 评分41

COPD:通过自然语言通信向黑盒教师模型做在线蒸馏

Communicative On-Policy Distillation: Distilling Black-Box Teacher Capabilities through Communication for Long-Horizon Tasks

AI 导读

研究者提出 Communicative On-Policy Distillation(COPD),用自然语言通信作为媒介,把不暴露 logits 的黑盒强模型能力蒸馏到更小的学生模型。

正文 · 原文

Authors:Lirui Luo, Kelong Mao, Xinwei Yang, Rongqing Li, Luyu Chen, Kieran Wong, Yudong Guo, Xinrui Wang, Jiayin Zhu, Gu Simiu, Sulong Xu, Cong Fang

View PDF

Abstract:On-policy distillation (OPD) is a powerful paradigm for improving language-model agents on long-horizon tasks, yet its conventional formulation relies on teacher logits that text-only black- box models do not expose. To address this limitation, we introduce Communicative On-Policy Distillation (COPD), which uses natural-language communication as a medium for distilling a stronger model's capabilities into a smaller student. During task execution, the student au- tonomously decides when to consult a black-box teacher and formulates open-ended natural- language requests without predefined question templates or intervention schedules. Successful student-teacher interactions are retained as privileged references to supervise fresh adviser-free student rollouts through on-policy self-distillation, jointly with task-reward reinforcement learning. On CAMG-II, COPD achieves 2.46 times the gain of the strongest black-box OPD baseline. The capabilities it learns through communication generalize to Terminal-Bench 2.1 and SWE-Bench Pro V2, improving task success without adviser access.
Comments: 13 pages, 8 figures, 2 tables. Project page: this https URL
Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
Cite as: arXiv:2611.23673 [cs.LG]
  (or arXiv:2611.23673v1 [cs.LG] for this version)
  https://doi.org/10.48550/arXiv.2611.23673

arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Lirui Luo [view email]
[v1] Wed, 18 Nov 2026 00:56:10 UTC (10,755 KB)

来源:arXiv cs.AI · arxiv.org