跳到正文
arXiv cs.AI· Jie Huang, Yuecheng Hong, Yao Chen, Shengding Jiao, Xudong Liu, Qiang Ye·· 4 小时前AI 评分38

OutfitterBench:评估 LLM 智能体的长周期资源分配与承诺协调能力

OutfitterBench: Evaluating LLM Agents on Long-Horizon Resource Allocation and Commitment Coordination

AI 导读

研究者推出 OutfitterBench,让 LLM 智能体经营户外装备租赁业务 200 天,需在固定资金上限下从 46 种产品目录采购并处理约 1,100 个预订请求。七款前沿 LLM 的实验显示,多个智能体表现不及简单规则基线,仅领先者能稳定超越它。分析发现部分智能体约束执行不一致、新决策与已有承诺冲突,且常错过更有利的资产配置机会。

正文 · 原文

View PDF HTML (experimental)

Abstract:In many long-horizon settings, an agent's own choices create future constraints: committing finite capacity establishes future obligations and forecloses later opportunities. Sustaining performance therefore requires both keeping evolving plans feasible and comparing the value of resource allocations with the alternatives they displace. Recent benchmarks increasingly evaluate LLM agents in continuing operations, but give less attention to how agents keep their own commitments feasible while reassessing resource allocations as opportunities change. We introduce OutfitterBench, where agents manage an outdoor-equipment rental business for 200 days, purchasing from a 46-product catalog and handling roughly 1,100 reservation requests. Under a fixed capital ceiling, reservations and uncertain walk-in demand compete for reusable equipment whose availability changes over time. Experiments with seven frontier LLMs show that several agents underperform a simple rule-based baseline, while only the leading agent consistently surpasses it. Trajectory analysis shows that some agents apply known constraints inconsistently and make new decisions that conflict with existing commitments. Agents with more reliable fulfillment can still secure low returns from comparable commitments or delay resource reallocation. The leading agent better connects preparation with economic choices but also misses opportunities for more profitable asset use. These findings highlight limitations in sustaining coherent resource plans and reassessing their value over time. The benchmark code is publicly available at this https URL.
Subjects: Artificial Intelligence (cs.AI)
Cite as: arXiv:2611.17933 [cs.AI]
  (or arXiv:2611.17933v1 [cs.AI] for this version)
  https://doi.org/10.48550/arXiv.2611.17933

arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Jie Huang [view email]
[v1] Fri, 13 Nov 2026 09:11:54 UTC (373 KB)

来源:arXiv cs.AI · arxiv.org