Metrics-in-the-Loop:复杂度反馈重塑智能体编写代码的质量
Metrics-in-the-Loop: Complexity Feedback Reshapes Agent-Written Code
AI 导读
一项 arXiv 研究(arXiv:2611.23817)提出 metrics-in-the-loop 推理时干预方法,在编码智能体的运行轨迹中注入自动计算的代码指标反馈:在智能体每次修改代码后,提示其圈复杂度超限的函数或未被测试覆盖的代码行,促使其及时重构或补强测试。在三个长周期编码基准和四个强模型上的实验显示,复杂度反馈能显著降低代码复杂度集中度(erosion),且不影响留出集上的正确性。
正文 · 原文
Abstract:Coding agents now write whole codebases in long autonomous runs, guided mainly by whether their tests pass. Yet passing tests alone does not reveal accumulating complexity or gaps in test coverage, which then persist as agents extend their own code. We introduce metrics-in-the-loop, an inference-time intervention that gives agents in-flight feedback: cheap, automatically computed code metrics injected into the agent's context during the trajectory. After each code-changing agent turn, we insert short feedback messages naming changed functions whose cyclomatic complexity exceeds a limit (complexity feedback) or changed lines that are skipped by tests (coverage feedback), giving the agent the opportunity to refactor code or strengthen tests before continuing. We run extensive experiments and ablations comparing several interventions across three long-horizon coding benchmarks and four strong models. We find that complexity feedback significantly reduces erosion, a measure of complexity concentration of code, without affecting held-out correctness of the code. Broadly, our experiments investigate a practical form of agent "re-railing", where timely, actionable measurements redirect agents before weaknesses become the foundation they keep building on. As agents scale to codebases with millions of lines, such interventions become critical to sustaining effective development and long-term code maintainability.
| Comments: | Agent Evaluation Science Symposium, Fall 2026, New York City |
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2611.23817 [cs.AI] |
| (or arXiv:2611.23817v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2611.23817 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Nicolas Chapados [view email]
[v1]
Wed, 18 Nov 2026 04:19:48 UTC (468 KB)
来源:arXiv cs.AI · arxiv.org