跳到正文
arXiv cs.AI· Simon Tang, Wenjie Shi, Yongjing Yin, Keheng Wang, Jialin Liu, Zheng Zhang, Linvo Lin, Xiangyuan Xue, Kan Ren, Jingang Wang, Xunliang Cai·· 4 小时前AI 评分52

AutoRL 论文提出面向自主强化学习的多智能体系统

AutoRL: A Multi-Agent System towards Autonomous Reinforcement Learning

AI 导读

作者发布 AutoRL 论文(arXiv:2611.17790),一个面向自主强化学习的多智能体系统,定位为未来递归自我改进(RSI)的基石。系统包含三类角色:Perceptors 监控训练动态并在异常时触发审议,三个 Proposers 诊断问题并细化干预方案,Orchestrator 审核后通过在线热更新或断点恢复落地变更。

正文 · 原文

Authors:Simon Tang, Wenjie Shi, Yongjing Yin, Keheng Wang, Jialin Liu, Zheng Zhang, Linvo Lin, Xiangyuan Xue, Kan Ren, Jingang Wang, Xunliang Cai

View PDF HTML (experimental)

Abstract:Recursive self-improvement (RSI) is one of the most promising paths toward an AI singularity for superintelligence. Achieving this vision requires AI systems to undertake the full training pipeline for iterative improvement. Reinforcement learning (RL) plays an essential role in this cycle to improve model capabilities and generalization, yet automating RL remains difficult due to unstable training dynamics and the need for sustained supervision. We introduce AutoRL, a multi-agent system for automating RL training, as a cornerstone for future RSI. Different from conducting research by end-to-end complete experiments, AutoRL monitors the evolving training state and integrates process-level supervision and research into an ongoing RL training, following the practical workflow of an RL researcher. Specifically, AutoRL consists of three RL-specific agentic roles. Perceptors track training dynamics, rollouts, evaluation, and efficiency, triggering deliberation when they detect anomalies, degradation, or stagnation. Three complementary Proposers diagnose problems and refine candidate interventions through shared evidence and checks. The Orchestrator reviews their recommendation and applies approved changes through online hot updates or checkpoint-based resumption. Experiments on Qwen3 and LongCat-Flash-Lite demonstrate the superiority of AutoRL over static RL recipes and a general agent. The research records show that existing agents can handle unattended RL supervision and dynamic adjustment. The difficulty for future RSI lies in developing fundamental methods that are valuable for resource-consuming validation.
Comments: 23 pages
Subjects: Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
Cite as: arXiv:2611.17790 [cs.AI]
  (or arXiv:2611.17790v1 [cs.AI] for this version)
  https://doi.org/10.48550/arXiv.2611.17790

arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Shengji Tang [view email]
[v1] Fri, 13 Nov 2026 06:06:36 UTC (4,201 KB)

来源:arXiv cs.AI · arxiv.org