可复现评测：生产级 Agent 安全提示词压缩的基础
Reproducible Evaluation as the Substrate for Safe Prompt Compression in Production Agents
AI 导读
一篇 NeurIPS 2026 Workshop 论文提出面向生产对话 Agent 的离线评测框架，可在其后端数据持续变化的情况下保持可复现，并支持失效后按需重新生成。
正文 · 原文
Abstract:Evaluating conversational agents that act on live, mutable enterprise data is hard: replayed sessions drift as backend state changes, and a benchmark built today can silently go stale as the data it describes moves on. We present an offline evaluation harness for a production conversational agent that stays reproducible despite the live data it operates on changing, and that can be regenerated on demand once it goes stale. We then use this harness as the fitness function for GEPA-based prompt optimization under a correctness gate that guards against reward hacking. Applied to a production merchant-facing agent, correctness-gated compression shrinks the agent's full prompt surface by 3.5-7.6% without regressing held-out task correctness, consistently across three independent subject-model families, and for one, correctness measurably improves.
| Comments: | 8 pages (4 pages main text plus references and appendices), 1 table. Accepted at the NeurIPS 2026 Workshop on Agents in the Wild |
| Subjects: | Artificial Intelligence (cs.AI) |
| ACM classes: | I.2.7; I.2.6 |
| Cite as: | arXiv:2610.31972 [cs.AI] |
| (or arXiv:2610.31972v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.31972 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Prabal Vashisht [view email]
[v1]
Thu, 22 Oct 2026 22:24:35 UTC (22 KB)
来源：arXiv cs.AI · arxiv.org