arXiv cs.AI· Jenny T. Liang, Mihika Bairathi, Wayne Chi, Ameet Talwalkar, Nishant Subramani, Valerie Chen·· 5 小时前AI 评分45
开发者如何编辑 AI 生成代码，以及 LLM 能多好地推断这些编辑？来自 53.6K 次开发者编辑的分析
How Do Developers Edit AI-Generated Code, and How Well Do LLMs Infer Their Edits? Insights from 53.6K Developer Edits
AI 导读
研究者发布 DECODE 数据集，收录来自 1000 多名开发者的 53.6K 次对 AI 生成代码的真实 IDE 内代码编辑，用于分析开发者何时、为何以及如何编辑 AI 代码。结果显示开发者编辑投入少、很快放弃 AI 生成代码；预测代码编辑对前沿模型和微调模型都是难题，前沿模型的编辑最贴近开发者意图但易过度编辑和回退人工修改，微调可提升相似度但仍有差距。
正文 · 原文
Abstract:Imperfections in AI-generated code require that software developers modify generations through manual code edits. However, little is known about how developers edit AI-generated code, and to what extent LLMs can infer implicit developer intent from this data, given the lack of high-quality, realistic code editing data. This information is essential for designing and building usable AI-developer interactions. To address this gap, we introduce DECODE (Developer Edits of Code Dataset), a dataset of 53.6K real-world in-IDE code edits of AI-generated code sourced from 1K+ developers. First, we use DECODE to analyze when, why, and how AI-generated code is edited. Developers invest little effort in editing code and abandon AI-generated code quickly, thereby motivating the need for LLMs to understand developer intents through their edits. Therefore, we next use DECODE to understand to what extent LLMs could infer developer edits. Predicting code edits is a challenging task for both frontier and fine-tuned models. Frontier models generate edits that best reflect developer intents, but are prone to over-editing and reverting developer-written edits. Fine-tuning LLMs can improve their ability to predict edits that are more similar to developers' as well as better reflect developer intent, but do not fully bridge the gap to frontier models. Together, our results demonstrate the unique insights on developer and LLM behavior that can be obtained through studying realistic code edit data to promote more better developer-AI interactions.
| Subjects: | Software Engineering (cs.SE); Artificial Intelligence (cs.AI); Human-Computer Interaction (cs.HC); Machine Learning (cs.LG) |
| Cite as: | arXiv:2607.25130 [cs.SE] |
| (or arXiv:2607.25130v3 [cs.SE] for this version) | |
| https://doi.org/10.48550/arXiv.2607.25130 arXiv-issued DOI via DataCite |
Submission history
From: Jenny Liang [view email]
[v1]
Mon, 27 Jul 2026 22:50:05 UTC (4,732 KB)
[v2]
Mon, 14 Sep 2026 16:31:50 UTC (4,732 KB)
[v3]
Tue, 27 Oct 2026 21:18:47 UTC (5,511 KB)
来源：arXiv cs.AI · arxiv.org