arXiv cs.AI· Shyam Agarwal, Olga Lvova, Agnia Sergeyuk, Anastasiia Serova, Timofey Bryksin, Thomas Zimmermann, Bogdan Vasilescu·· 3 小时前AI 评分54
JetBrains 实证研究：工程师的 token 消耗衡量什么
The AI Era's Lines of Code? A Case Study on What an Engineer's Token Spend Measures
AI 导读
一项研究将 JetBrains 404 名工程师五周的编码智能体 token 用量与其 commits 关联，并结合 412 人调查分析 token 消耗能衡量什么。
正文 · 原文
Abstract:Organizations use engineers' token spend to monitor AI adoption, set budgets, and measure productivity, but the metric's relationship with engineering output is unclear. We link five weeks of token usage in coding agents to the commits of 404 engineers at JetBrains, and analyze a company survey of 412 employees. We find that token spend varied mostly between engineers, and which engineers spent the most stayed largely the same over the five weeks. Among weeks with similar commit counts, the 90th percentile of spend was 39 to 240 times the 10th. Within an engineer, doubling spend was associated with about 10% more commits. Available work metrics explained 38% of the variation in spend between engineers and 13% of the week-to-week variation within an engineer. In the survey, respondents in technical roles reported in nearly nine of ten agent evaluations using agents for tasks beyond coding, such as analysis, research, and planning. Token spend therefore measures how much engineers use agents across many kinds of work and not the outcome and value of that work. We recommend using token spend at most for resource accounting, and evaluating agent-assisted work with evidence about tasks and outcomes.
| Subjects: | Software Engineering (cs.SE); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2611.13396 [cs.SE] |
| (or arXiv:2611.13396v1 [cs.SE] for this version) | |
| https://doi.org/10.48550/arXiv.2611.13396 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Shyam Agarwal [view email]
[v1]
Tue, 10 Nov 2026 00:17:26 UTC (306 KB)
来源：arXiv cs.AI · arxiv.org