AgentTrustCI：面向 Java LLM 智能体的构建时策略、溯源与轨迹测试框架
AgentTrustCI: Build-Time Policy, Grounding, and Trajectory Tests for Java LLM Agents
AI 导读
AgentTrustCI 是一个基于 Java/JUnit 5 的框架，将 LLM 智能体的策略、溯源与轨迹预期表达为基于录制轨迹的可重复构建时测试。在覆盖 7 个企业领域的 55 个合成案例上，完整流水线正确分类全部 30 个注入故障和 25 个通过控制项，12 个关键案例零遗漏；LLM-judge 为可选语义评估扩展点。该成果是 Java 发布管线中表达行为契约的实用模式。
正文 · 原文
Abstract:Infrastructure security does not establish whether an LLM agent selected an allowed tool, grounded its answer in tool output, or followed a required workflow. AgentTrustCI is a Java/JUnit 5 framework that expresses these application-layer expectations as policy, grounding, and trajectory checks over recorded traces. A deterministic replay implementation supports repeatable build-time tests; a judge interface provides an extension point for optional semantic evaluation. On 55 synthetic cases spanning seven enterprise domains, the full pipeline correctly classifies all 30 injected faults and 25 passing controls, including ten non-trivial controls, and misses none of 12 designated critical cases. Each single-layer configuration correctly classifies 35 cases because the corpus deliberately isolates ten faults per layer. A grounded answer obtained through a disallowed sequence illustrates the additional coverage of trajectory checks. Repeated measurements quantify validator execution separately from process startup and reporting. These results establish coverage of the authored regression suite, not production accuracy. We describe a production-trace validation protocol, optional LLM-judge integration, and maintenance of expectations as workflows evolve. The contribution is a practical pattern for expressing behavioral contracts within an existing Java release pipeline.
| Comments: | Accepted for publication in the 2026 International Conference on Computer and Applications (ICCA) |
| Subjects: | Software Engineering (cs.SE); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2611.06674 [cs.SE] |
| (or arXiv:2611.06674v1 [cs.SE] for this version) | |
| https://doi.org/10.48550/arXiv.2611.06674 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Sachin Gupta [view email]
[v1]
Wed, 4 Nov 2026 15:46:11 UTC (82 KB)
来源：arXiv cs.AI · arxiv.org