跳到正文
arXiv cs.AI· Disha Sheshanarayana, Sai Linisha Reddy Goli·· 4 小时前AI 评分50

验证器判断能否跨抽象层存活?面向 LLM Agent 评测的 Verification Transport 测量

Do Verifier Judgments Survive Abstraction? Measuring Verification Transport for LLM Agent Evaluation

AI 导读

一项 arXiv 研究提出 Verification Transport 测试,衡量 LLM 验证器在抽象边界下判断是否稳定。在 4 个 agent 评测基准、7 个验证器骨干、4 种协议和 3 种抽象边界上,发现显著的判断失效,宏观抽象与委派下达约 31%;恢复执行溯源可挽回 85.5% 的失效。结果表明仅凭原始视图准确率不足以刻画验证器可靠性。

正文 · 原文

View PDF HTML (experimental)

Abstract:LLM-based verifiers are increasingly used to evaluate agent executions, but their reliability is typically measured under a single representation of the trajectory. In deployed agent systems, the same execution may be exposed through composed tools, delegation boundaries, or compacted histories, changing what execution evidence is directly observable to the verifier. It therefore remains unclear whether correct verifier judgments persist across such abstraction boundaries. We introduce Verification Transport, a reliability test that measures judgment stability under abstraction-induced changes in execution observability. We construct matched native, abstracted, and restored views of the same frozen executions while holding the task, realized trajectory, outcome, ground truth, evaluation criterion, and verifier fixed. Across four agent-evaluation benchmarks, seven verifier backbones, four verification protocols, and three abstraction boundaries, we find substantial transport failure, reaching about 31% under macro abstraction and delegation. Verification protocols exhibit distinct failure modes: direct verification more often produces incorrect verdicts, whereas evidence-grounded and deliberative protocols tend to abstain under limited observability. This suggests that lower transport failure rates do not necessarily imply greater reliability. Restoring the corresponding execution provenance recovers 85.5% of transport failures, while recovery is lower when provenance is lossy or unavailable. These results show that native-view accuracy alone is insufficient to characterize verifier reliability when execution evidence crosses abstraction boundaries.
Comments: Accepted at NeurIPS 2026 Workshop on Who Verifies the Agents? Toward Reliable Agent Development
Subjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
Cite as: arXiv:2611.18471 [cs.CL]
  (or arXiv:2611.18471v1 [cs.CL] for this version)
  https://doi.org/10.48550/arXiv.2611.18471

arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Disha Sheshanarayana [view email]
[v1] Fri, 13 Nov 2026 17:44:01 UTC (2,005 KB)

来源:arXiv cs.AI · arxiv.org