跳到正文
arXiv cs.AI· Imene Kerboua, Fatemeh Pesaran Zadeh, Xing Han L\`u, Weijian Qi, Alexander Miller, Junyi Song, Yunjia Tian, Dongjin Kang, Seyeon Choi, Marzia Nouri, Ewen Gueguen, Matteo Boglioni, Fengyuan Liu, Zeyi Liao, Mengqi Yuan, Yue Li, Alexandre Lacoste, Alexandre Drouin, Spandana Gella, Huan Sun, Gunhee Kim, Siva Reddy·· 4 小时前AI 评分42

WebArena-Pro:面向 Web Agent 的异构多模态可复现基准

WebArena-Pro: A Heterogeneous, Multimodal, Reproducible Benchmark for Web Agents

AI 导读

研究者推出 WebArena-Pro 基准,包含 300 个任务、20 个自托管 Web 应用,覆盖六个领域类别,并将音频和视频作为核心观测模态,支持可复现的并行评测。

正文 · 原文

Authors:Imene Kerboua, Fatemeh Pesaran Zadeh, Xing Han Lù, Weijian Qi, Alexander Miller, Junyi Song, Yunjia Tian, Dongjin Kang, Seyeon Choi, Marzia Nouri, Ewen Gueguen, Matteo Boglioni, Fengyuan Liu, Zeyi Liao, Mengqi Yuan, Yue Li, Alexandre Lacoste, Alexandre Drouin, Spandana Gella, Huan Sun, Gunhee Kim, Siva Reddy

View PDF HTML (experimental)

Abstract:Web agents powered by large language and vision-language models are increasingly applied to realistic browser work that spans heterogeneous applications, multimodal content, and stateful workflows. However, existing reproducible web-agent benchmarks cover only a small number of web applications drawn from a few software categories, and restrict modality to text and vision. Live benchmarks broaden site coverage but sacrifice reproducibility, since pages and data drift between runs. Moreover, existing benchmarks do not meaningfully evaluate whether agents can understand and use audio and video content embedded within web tasks. To address these gaps, we introduce WebArena-Pro, a benchmark comprising 300 tasks across 20 self-hosted web applications in six domain categories, spanning distinct interface conventions, workflows, and data models. Across the evaluated agents, the best performance is achieved by both Codex with GPT 6.1 Sol and Claude Code with Claude Opus 5.5 at 62.3%, followed by OpenCode with Kimi 3.1 Pro at 60.7%. The strongest GenericAgent configurations, Gemini 3.1 Pro omni and Gemma4 31B, both achieve 30.3%. Among reproducible, human-curated web agent benchmarks, WebArena-Pro provides the broadest application coverage and the most comprehensive multimodal support to date. The benchmark treats audio and video as core observations alongside text and vision, with dedicated actions for extracting information from each. WebArena-Pro runs each task in isolation and supports reproducible, parallel evaluation. Tasks are authored through a dedicated annotator interface, filtered by LLM-assisted triage, and finally validated by humans before release. Project page: this https URL
Comments: Accepted at NeurIPS 2026 E&D Track
Subjects: Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Machine Learning (cs.LG)
Cite as: arXiv:2611.17955 [cs.AI]
  (or arXiv:2611.17955v1 [cs.AI] for this version)
  https://doi.org/10.48550/arXiv.2611.17955

arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Imene Kerboua [view email]
[v1] Fri, 13 Nov 2026 09:44:48 UTC (782 KB)

来源:arXiv cs.AI · arxiv.org