Mind the Gap:TriAgent 多智能体系统实现文档持续维护
Mind the Gap: A Deployed Multi-Agent System for Documentation Maintenance
AI 导读
一个以"TriAgent"分诊智能体为核心的多智能体系统被部署用于持续识别并修复文档滞后。43 天影子运行中处理 499 张工单、发现 92 处文档缺口,81.1% 的可验证断言正确;修复后模型再答同一工单时,盲评在 71.4% 的案例中更偏好修复后的回答。该系统已被 IAAI-27 接收。
正文 · 原文
Abstract:The "AI revolution" has enabled teams across the technology industry to deliver software features at unprecedented velocity. While code evolves faster than ever, customer-facing documentation often struggles to stay up to speed. As users increasingly rely on LLM-based agents that consume documentation as their primary interface, the impact of this documentation gap worsens: stale information propagates and is treated as authoritative. Signals for identifying where documentation has fallen behind already exist in abundance, both explicit (e.g., resolved support tickets revealing what documentation lacks) and implicit (e.g., code-documentation discrepancies detectable from automated loops). We deploy a multi-agent system, built around a triage agent we call "TriAgent," that continuously extracts these signals and repairs documentation drift. Over 43 days of shadow-mode operation, our system triaged 499 tickets and surfaced 92 gaps in the documentation. Using a panel of LLM judges from independent model families, 81.1% of TriAgent's verifiable assertions were correct (245 of the 302 assertions the human resolution addresses; majority of three judges); on length-comparable pairs the panel preferred TriAgent's response in 70.5% of pairs and the human engineer's in 23.2% (the rest had no judge majority). Importantly, agent- and human-sourced replies tended to be complementary, with the human often surfacing insights that were either buried within the existing documentation or missing from it entirely; our system turns each such insight into a documentation gap and, after human review, a fix. Repairing a gap measurably improved the agent's later answer: when the agent re-answered the same ticket after the repair, blinded judges preferred the post-repair answer over its own pre-repair answer 71.4% of the time (n=35). A blinded human audit confirms strong alignment on self-improvement.
| Comments: | Accepted at IAAI-27 (Innovative Applications of AI, co-located with AAAI 2027), Emerging Applications track. 12 pages (6 pages + references and appendix), 2 figures, 6 tables |
| Subjects: | Software Engineering (cs.SE); Artificial Intelligence (cs.AI); Computation and Language (cs.CL) |
| Cite as: | arXiv:2611.23142 [cs.SE] |
| (or arXiv:2611.23142v1 [cs.SE] for this version) | |
| https://doi.org/10.48550/arXiv.2611.23142 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Lorenzo Masoero [view email]
[v1]
Tue, 17 Nov 2026 17:59:13 UTC (45 KB)
来源:arXiv cs.AI · arxiv.org