arXiv cs.AI· Keyun Zhang, Dapeng Yin, Fanfu Wang, Shuang Lang, Zhenyu Zhu, Yulin Huang, Chong Yang, Pengjie Wang·· 2 小时前AI 评分35
ATLAS-Bench：面向本地生活搜索的智能体工具使用基准
ATLAS-Bench: An Agentic Tool-Use Benchmark for Local Area Search
AI 导读
ATLAS-Bench 是一个评估 LLM 智能体本地搜索能力的基准，包含基于真实搜索事件构建的 1,200 条查询，覆盖三个地区，区分强制性要求与偏好、事实判断与主观评价，并配套标准化评测环境 ATLASBox。
正文 · 原文
Abstract:Reliable local search is essential for LLM agents to support everyday decisions and longer plans by finding places that meet users' needs at given times and locations. Yet existing benchmarks provide limited coverage of whether search results satisfy mandatory requirements, account for preferences, and ground factual and experiential judgments in place attributes and reviews. We introduce ATLAS-Bench, a benchmark of 1,200 queries across three regions, expanded from real search events using place attributes, reviews, and existing user profiles from the platform. Each query distinguishes mandatory requirements from preferences and factual conditions from subjective judgments, with supporting evidence and criteria for checking each requirement. Queries are validated through iterative assertion checks and cross-model review. We also introduce ATLASBox, a standardized evaluation environment with search tools and a fixed point-of-interest (POI) knowledge base. Given a query, timestamp, and location, an agent searches place attributes and reviews to recommend evidence-supported options or report that no place meets all mandatory requirements. We evaluate answer quality, trajectory quality, and execution cost. Across 14 configurations of 11 models, GPT-6-Luna achieves the highest constraint satisfaction rate among returned recommendations (71.5%) and factual accuracy (95.6%). However, it returns at least one acceptable recommendation for fewer satisfiable queries than GLM-5.3 (67.3% versus 85.3%). These results highlight the challenge of satisfying individual requirements while providing usable recommendations across queries.
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.39438 [cs.AI] |
| (or arXiv:2610.39438v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.39438 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Keyun Zhang [view email]
[v1]
Thu, 29 Oct 2026 06:14:52 UTC (1,761 KB)
来源：arXiv cs.AI · arxiv.org