arXiv cs.AI· Wenbo Huang, Jianxin Bi, Shunyu Yang, Ruiyang Si, Rui Ni, Qiang Wang, Shulong Jiang, Yintianrun Zhang, Feixiang Ruan, Kewei Zhang, Ye Huang, Shukai Gong, Daquan Zhou·· 4 小时前AI 评分29
具身自改进智能体研究：智能体学的是任务还是验证标准？
Learning the Task or Learning the Verifier? Rethinking Criteria Exposure in Embodied Self-Improvement
AI 导读
arXiv 论文提出 Criteria Exposure（CE）概念，指将机器人仿真基准的成功标准暴露给学习智能体的做法。对照实验显示，CE 下智能体代码在给定标准下成功率快速提升，但标准稍有改动（仍不偏离原指令）优势即消失，且学到的技能跨任务迁移差。作者进一步提出 instruction-faithful criteria，能更好反映智能体生成程序的真实表现。
正文 · 原文
Authors:Wenbo Huang, Jianxin Bi, Shunyu Yang, Ruiyang Si, Rui Ni, Qiang Wang, Shulong Jiang, Yintianrun Zhang, Feixiang Ruan, Kewei Zhang, Ye Huang, Shukai Gong, Daquan Zhou
Abstract:Self-improving embodied agents are increasingly being developed and compared on robot simulation benchmarks. Several recent works expose the benchmark's success criteria to the learning agent, a practice we call Criteria Exposure (CE). These criteria, originally written to verify policies trained by imitation learning, often demand more than the corresponding language instructions, so a score from such a verifier may not faithfully reflect whether the instructed task was completed, especially for robot-control programs written by an agent. As embodied coding agents report ever higher benchmark scores, a natural question arises: when a coding agent is exposed to the benchmark's criteria, does it generate code that better performs the instructed task, or code that simply overfits to the verification criteria? We conduct controlled experiments with a self-improving agent with and without CE, and find that with CE the agent quickly produces code with a higher success rate under the given criteria, but this advantage disappears when the criteria are changed slightly without departing from the original instructions. More importantly, the skills the agent develops under CE transfer poorly across tasks. We further propose instruction-faithful criteria and show that they better reflect the performance of agent-generated programs.
| Comments: | 22 pages, 8 figures, 10 tables. Project page: this https URL |
| Subjects: | Artificial Intelligence (cs.AI); Robotics (cs.RO) |
| Cite as: | arXiv:2611.08318 [cs.AI] |
| (or arXiv:2611.08318v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2611.08318 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Wenbo Huang [view email]
[v1]
Thu, 5 Nov 2026 17:14:37 UTC (2,921 KB)
来源：arXiv cs.AI · arxiv.org