跳到正文
arXiv cs.AI· Ibragim Badertdinov, Alexander Golubev·· 4 小时前AI 评分51

编码智能体行为轨迹解读研究发布,分析143,209条执行轨迹

Interpreting Coding Agents Through Their Behavioral Trajectories

AI 导读

一项arXiv研究(arXiv:2611.17440)分析143,209条执行轨迹,覆盖125种agent配置和117个模型,通过规则解析器把执行轨迹映射为仓库搜索、文件编辑、测试执行等高层动作。

正文 · 原文

View PDF HTML (experimental)

Abstract:Standard coding-agent benchmarks primarily evaluate binary task success, treating execution as a black box and obscuring the behaviors that produce those outcomes. We analyze a large corpus of 143,209 execution trajectories from 125 agent configurations spanning 117 unique models. To enable comparisons across heterogeneous agent interfaces, we develop a rule-based parser that maps raw execution traces to high-level actions such as repository search, file editing, and test execution, and audit its predictions against a six-model consensus reference on 500 events. We find that agents with similar solve rates can nevertheless differ substantially in how they explore, modify and verify code. Moreover, without using task outcomes, token counts, or tool names as features, aggregate behavioral profiles retrieve the corresponding agent stack across disjoint repository sets and programming languages, although task composition is not fully controlled in the cross-repository setting. Through analyses of frontier models and agent scaffolds, we further show that scaffold choice and reasoning effort systematically shift agents' search, editing, and testing behavior. We release the deterministic parser together with the full annotation and analysis pipeline alongside an annotated trajectory subset.
Comments: Accepted at the 1st Workshop on Interpreting Agent Behavior (IAB) at NeurIPS 2026. Code: this https URL
Subjects: Software Engineering (cs.SE); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
Cite as: arXiv:2611.17440 [cs.SE]
  (or arXiv:2611.17440v1 [cs.SE] for this version)
  https://doi.org/10.48550/arXiv.2611.17440

arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Ibragim Badertdinov [view email]
[v1] Thu, 12 Nov 2026 19:11:23 UTC (1,411 KB)

来源:arXiv cs.AI · arxiv.org