arXiv cs.AI· Qiming Shi, Yikun Li, Beidi Luan, Rui Sun, Yan Ma, Jing Li, Daxin Jiang, Zuo Bai·· 3 小时前AI 评分37
AutoVend：评测 LLM 智能体长周期自动售货机运营的基准
AutoVend: Benchmarking LLM Agents on Long-Horizon Autonomous Vending Operations
AI 导读
AutoVend 基准让一个 LLM 智能体通过网页操作界面管理分布在五类站点的 20 台售货机，模拟运营 180 天，模拟器包含共享仓库、采购提前期、付费补货、库存过期和持续运营成本。五个被评测的 LLM 智能体中四个不及规则库存控制基线，且按单位推理成本收益排序与单独收益排序不同。轨迹分析显示智能体在可行动作落地、库存承诺管理和将反馈转化为策略修订方面存在困难。
正文 · 原文
Abstract:Large language model (LLM) agents are extending agent-mediated commerce on the Web from shopping for consumers to operating businesses for sellers. In online-to-offline (O2O) commerce, decisions are issued through Web-based platforms but realized through costly physical execution. Evaluating such agents requires going beyond individual transactions to ask whether decisions made online turn into sustained economic returns. We introduce AutoVend, a benchmark in which one LLM agent manages 20 vending machines across five site types through a Web-based operating interface for 180 simulated days. Its data-grounded simulator combines a shared warehouse, procurement lead times, paid replenishment, inventory expiration, and recurring operating costs. We evaluate five LLM agents against a rule-based inventory-control baseline and diagnose their site coordination, execution, and performance over time. Four of five evaluated LLM agents fall short of this baseline, and returns per dollar of inference cost rank models differently from returns alone. Operational analyses show that gains in individual metrics do not consistently yield stronger economic growth, as inventory and execution costs erode these gains. Trajectory analysis further reveals difficulties in grounding feasible actions, managing inventory commitments, and converting feedback into effective policy revisions.
| Subjects: | Artificial Intelligence (cs.AI); Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.36980 [cs.AI] |
| (or arXiv:2610.36980v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.36980 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Zuo Bai [view email]
[v1]
Tue, 27 Oct 2026 10:33:06 UTC (596 KB)
来源：arXiv cs.AI · arxiv.org