LLM 智能体在公共服务场景中会做什么：读档案但常自行其是的实证研究
Read and Ignore: What Self-Directed LLM Agents Do in Public Service Scenarios
AI 导读
一项 arXiv 研究构建四个经专家验证的公共服务场景，收集 10 个 LLM 在四种提示条件下的 2,400 条轨迹，并与 1,000 次无上下文沙盒运行对比。智能体在 96.9% 的开放式运行中读取记录，但仅 18.7% 产出结果或自选任务；给定明确任务时产出率达 86.0%。领域上下文使边界试探轮次减少 60%，部分产出仍与记录矛盾，研究团队发布了可供机构采购前自测的场景、代码与轨迹。
正文 · 原文
Abstract:Underspecified instructions leave LLM agents to decide what to do. In public services, staff write these instructions, but the agent's decisions affect residents, patients, and students. We build four expert-validated public-service scenarios with records and decision levers that can be checked in the files, collect $2{,}400$ trajectories from ten LLMs under four prompt conditions, and compare them with $1{,}000$ runs in a context-free sandbox. Agents read the records in $96.9\%$ of open-ended runs and produce an artifact or a self-chosen task in $18.7\%$; with an explicit task, $86.0\%$ produce. Domain context does not change how many runs produce ($18.7\%$ vs.\ $16.0\%$), and the production ranking of the models largely carries over (Spearman $0.79$), while the ranking by whether a model reads the records does not ($0.19$). Context cuts boundary-probing turns per run by $60\%$, and some artifacts contradict the records. We contribute a test that a public-service institution can run on its own records before procurement; its rule-based taxonomy needs no labeled data, and scenarios, code, and trajectories are released.
| Comments: | Trustworthy AI for Good @ NeurIPS 2026 |
| Subjects: | Artificial Intelligence (cs.AI); Human-Computer Interaction (cs.HC); Multiagent Systems (cs.MA) |
| ACM classes: | H.1.2 |
| Cite as: | arXiv:2611.15460 [cs.AI] |
| (or arXiv:2611.15460v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2611.15460 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Diana Kozachek [view email]
[v1]
Wed, 11 Nov 2026 13:31:32 UTC (3,067 KB)
来源：arXiv cs.AI · arxiv.org