GUI 智能体自适应训练新框架：从哪里学、回滚多远才高效
Where to Learn and How Far to Roll Out: Efficient Adaptive Training for GUI Agents
AI 导读
一篇 arXiv 论文提出面向 GUI 智能体的两阶段自适应训练框架，同时解决“在哪里学”和“回滚多远”两个问题。方法用动作分歧度从专家轨迹中选出可复用的中间检查点，Stage I 在这些 pivot 上做单步 RL，Stage II 按任务进度自适应扩展 Local、Suffix、Root 训练范围。
正文 · 原文
Abstract:Trajectory-level GRPO is a common approach for training multi-step GUI agents, but repeatedly rolling out complete trajectories from the task root can be inefficient: task-level rewards provide coarse supervision over which intermediate decisions require improvement, while already-mastered prefixes are repeatedly re-executed before reaching difficult states. We introduce a two-stage adaptive training framework that jointly addresses where to learn and how far to roll out. We first construct reusable intermediate checkpoints from successful expert trajectories and select informative pivots using action disagreement, which measures uncertainty directly from sampled GUI actions. Stage I performs one-step RL at these pivots to improve uncertain local decisions. Stage II then progressively expands policy responsibility through adaptive Local, Suffix, and Root training according to task progress. Recoverable checkpoints enable repeated training from intermediate states without replaying the preceding trajectory. Across MobileGym and CUA-Gym-Hub, our method reduces environment interaction by 56-61% relative to Vanilla GRPO while improving task performance, demonstrating that jointly allocating training across states and rollout ranges can substantially improve GUI-agent RL efficiency.
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.39251 [cs.AI] |
| (or arXiv:2610.39251v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.39251 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Yuxiang Chai [view email]
[v1]
Thu, 29 Oct 2026 03:03:29 UTC (1,880 KB)
来源：arXiv cs.AI · arxiv.org