SkillShift-Bench：首个评估 Web 智能体技能学习在环境演化下表现的新基准
SkillShift-Bench: Benchmarking Skill Learning under Environment Evolution for Web Agents
AI 导读
arXiv 论文提出 SkillShift-Bench，这是首个评估技能学习在开放环境受控变化下表现的新基准，已入选 EMNLP 2026 主会。
正文 · 原文
Abstract:Skill learning enables web agents to reuse prior experience for long-horizon interactive tasks. However, learned skills are typically evaluated in closed environments, overlooking the fact that real-world web deployment involves continuous shifts in invocation context, perceptual grounding, and execution feasibility. To this end, we introduce SkillShift-Bench, the first benchmark for evaluating skill learning with controlled open-environment shifts. SkillShift-Bench simulates Contextual Shift, Perceptual Shift, and Execution Shift across three self-hosted web systems, and evaluates agents with diagnostics covering task success, interaction cost, skill use, and skill failure. Comprehensive experiments reveal systematic robustness degradation under controlled shifts, characterized by lower success rates, higher interaction costs, and more frequent skill failures, with Execution Shift exerting the strongest pressure. We further formulate shifted skill execution as a nonstationary local executability problem and provide both a dynamic-regret analysis and a practical test-time adaptation method, demonstrating the feasibility of improving learned-skill execution without relearning the skill library. Code, tasks, environments, documentation, and the leaderboard are available at this https URL.
| Comments: | 70 pages, 8 figures. Accepted to EMNLP 2026 (Main Conference) |
| Subjects: | Artificial Intelligence (cs.AI); Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.33528 [cs.AI] |
| (or arXiv:2610.33528v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.33528 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Bo-Yu Chen [view email]
[v1]
Sat, 24 Oct 2026 08:09:00 UTC (6,810 KB)
来源：arXiv cs.AI · arxiv.org