arXiv cs.AI· Pengcheng Zhou, Xuanyu Liu, Yanchen Yin, Tianjie Ju, Bobo Li, Shengqiong Wu, Hao Fei, Mong-Li Lee, Wynne Hsu·· 1 小时前AI 评分34
LiveGeo 发布：持续演化地理环境中的智能体基准测试
LiveGeo: Benchmarking Geographic Agents in Continuously Evolving Worlds
AI 导读
研究者推出可控且持续演化的地理环境平台 GeoWorld，以及基于该平台的 LiveGeo 基准，用于评测 LLM 智能体在真实地理环境中随世界演化的可靠决策能力。结果显示，即便是前沿模型在静态任务上也表现吃力，没有模型在全部任务中持续领先，且静态表现好不等于动态可靠性。分析发现关键瓶颈在于智能体难以获取证据并转化为有效决策，且难以随世界演化维持决策正确性。
正文 · 原文
Abstract:We introduce GeoWorld, a controllable and continuously evolving geographic environment platform for LLM agents, and LiveGeo, a benchmark for evaluating reliable decision-making in real-world geographic environments using GeoWorld. Unlike existing evaluations that largely reason and plan over fixed geographic states, GeoWorld is grounded in real-world geographic structure and allows external world states to evolve along their own temporal processes while preserving verifiable state histories. LiveGeo places geographic tasks within this evolving environment, requiring agents to actively acquire task-relevant information, verify whether the evidence supporting prior decisions remains valid, and revise their plans when necessary. Our results reveal that even frontier LLMs struggle with static tasks, with no model consistently excelling across tasks, while strong static performance does not ensure dynamic reliability. Further analysis identifies a key bottleneck: agents struggle to acquire and translate relevant evidence into valid decisions and maintain their correctness as the world evolves. By releasing LiveGeo together with its GeoWorld platform, we aim to provide the community with a flexible and extensible infrastructure for studying the next generation of agents capable of reliable action in evolving geographic worlds.
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2611.17267 [cs.AI] |
| (or arXiv:2611.17267v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2611.17267 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Pengcheng Zhou [view email]
[v1]
Thu, 12 Nov 2026 18:45:58 UTC (44,615 KB)
来源：arXiv cs.AI · arxiv.org