跳到正文
arXiv cs.AI· Zelin He, Siyu Zhu, Jacob Helwig, Yujian Liu, Li An, Rohit Jain, Shiyu Chang, Alborz Geramifard·· 4 小时前AI 评分45

ModHarness-RL:将 harness 重构为可学习、可演化的程序用于智能体 RL

ModHarness-RL: Recasting Harnesses as Learnable and Evolving Programs for Agentic RL

AI 导读

ModHarness-RL 将智能体强化学习(RL)中固定不变的 harness 重构为从数据学习、随模型训练演化的程序。它把交互记录为结构化 State,由 Compression 程序组装历史上下文、Steering 程序在行为偏离时提供引导;训练中候选程序从近期轨迹提出并经执行验证。

正文 · 原文

View PDF HTML (experimental)

Abstract:Agentic reinforcement learning (RL) increasingly relies on a harness, the scaffold around the model that decides what it sees of its own interaction and what guidance it receives. Most existing approaches train under a hand-written harness that stays fixed throughout. However, an effective harness depends on the domain, the data and the model, and keeps changing as the model trains. We introduce ModHarness-RL, which recasts the harness as programs that are learned from data and evolve as the model trains with RL. ModHarness-RL records the interaction as structured State, a set of typed blocks, and two core components of the harness are programs over these blocks, a Compression program that assembles the accumulated history into the model's context and a Steering program that supplies guidance when behavior deviates. During RL training, candidate programs are proposed from recent trajectories, verified by execution, and replayed over past trajectories, so the harness tracks how the model's behavior shifts. We study CodeSearch in depth and further analyze ALFWorld and BIRD SQL, with models from 1.5B to 8B. The proposed Compression and Steering mechanisms improve held-out performance over the fixed-harness baseline in most settings. These results suggest that a harness for agentic RL can be learned from the task's data and evolved with the model.
Subjects: Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Machine Learning (cs.LG)
Cite as: arXiv:2611.18533 [cs.AI]
  (or arXiv:2611.18533v1 [cs.AI] for this version)
  https://doi.org/10.48550/arXiv.2611.18533

arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Zelin He [view email]
[v1] Fri, 13 Nov 2026 18:45:49 UTC (573 KB)

来源:arXiv cs.AI · arxiv.org