跳到正文
arXiv cs.AI· Jingxing Wang, Teng Wang, Xingyu Lou, Zihan Guo, Jun Wang, Weiwen Liu, Weinan Zhang, Jianghao Lin·· 2 小时前AI 评分41

训练 GUI Agent 需要多完整的环境?MobileGym 逐维剖析 reward、数据与 feature coverage

How Complete Must an Environment Be to Train a Capable GUI Agent?

AI 导读

研究者提出 MobileGym 框架,沿 reward coverage、data coverage、feature coverage 三个维度拆解 GUI Agent 训练环境的完整度,并用 Qwen3-VL-4B-Instruct 做控制实验。

正文 · 原文

View PDF HTML (experimental)

Abstract:Graphical user interface (GUI) agents are widely trained in executable GUI environments. To lower cost, build environments in batches, and reduce manual effort, each work trades away part of the environment through its construction choices. We describe these choices along three dimensions: reward coverage, the goal-reaching trajectories the verifier rewards; data coverage, the application data the agent is exposed to beyond what the task reads or writes; and feature coverage, the application features beyond what the task uses. Existing works set each dimension on engineering grounds, but they do not measure its effect on training. How complete must an environment be to train a capable agent, and along which dimensions? In MobileGym, we adjust each dimension separately, train Qwen3-VL-4B-Instruct, and evaluate every policy in the original full environment. A verifier that rewards only one reference path lowers success by 10.53 points, and removing unneeded records also lowers success. Pruning unneeded features instead raises success from 51.61% to 56.13%, entirely on tasks whose environments it left unchanged. The gain reverses when each episode is weighted equally and vanishes when only episodes on the pruned tasks are balanced. Without pruning, length weighting there reinforces long detour successes that erode skills on other tasks. Following these findings, we build MobileGym-Lite, which keeps reward and data coverage complete and prunes unneeded features. It outperforms Full in the full environment (55.48% versus 51.61% success) with lower construction cost.
Comments: 15pages, 6 figures
Subjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
Cite as: arXiv:2611.23946 [cs.CV]
  (or arXiv:2611.23946v1 [cs.CV] for this version)
  https://doi.org/10.48550/arXiv.2611.23946

arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Jingxing Wang [view email]
[v1] Wed, 18 Nov 2026 06:32:44 UTC (479 KB)

来源:arXiv cs.AI · arxiv.org