预算配对反事实审计:视觉智能体可信评估新方法
Choosing What to Observe: Budget-Paired Counterfactual Audits for Credible Evaluation of Visual Agents
AI 导读
一项 arXiv 研究提出“预算配对反事实审计”评估协议,用于可信评估需要分配有限观察预算读取密集文档的视觉智能体。该协议在固定公开证据与区域绑定的前提下,仅扰动剩余预算或声明任务参数,并将预期准确率损失分解为逐步获取损失与终端决策损失,区分规划与固定角色启发式。实验覆盖四十个配对路由锚点与五个开源权重模型。
正文 · 原文
Abstract:Visual agents that read dense documents must ration a finite observation budget across candidate regions, yet answer accuracy conflates where an agent looks, how it adapts to the reads that remain, and how it an?swers once observation stops. We introduce budget-paired counterfactual audits, an evaluation protocol that holds public evidence and region bindings fixed while perturbing only the remaining budget or de?clared task parameters, and credits every acquired history with exact continuation values. The resulting accounting decomposes lost expected accuracy into stepwise acquisition loss and terminal decision loss, separating planning from fixed-role heuristics. Across forty paired routing anchors, five open-weight
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2611.24867 [cs.AI] |
| (or arXiv:2611.24867v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2611.24867 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Ruiqi Liu [view email]
[v1]
Wed, 18 Nov 2026 18:20:12 UTC (1,575 KB)
来源:arXiv cs.AI · arxiv.org