跳到正文
arXiv cs.AI· Boxi Zhou, German Sviridov, Dmitrii Ustiugov, Haris Javaid·· 1 小时前AI 评分46

用超时级联降低编程智能体推理成本

Reducing Inference Cost of Coding Agents with Timeout Cascades

AI 导读

一项 arXiv 研究评估了 2024 年以来发布的 36 个开源大小模型在软件工程基准上的表现,发现小模型在简单和中等复杂度任务上准确率已接近大模型,差距集中在少数困难任务上。

正文 · 原文

View PDF HTML (experimental)

Abstract:AI coding agents have become one of the most popular applications of large language models. Yet, agentic AI comes at the cost of a tremendous amount of GPU compute resources, which limits its profitability and its viability at scale. In this paper, we investigate whether smaller, cheaper models can cut this cost without sacrificing accuracy.
We evaluate 36 popular large and small open-weight models released since 2024 on a software engineering benchmark, and find that recent small models are closing the accuracy gap compared to their large counterparts. We investigate the structure of this gap and find that the gap is highly concentrated around hard tasks that small language models fail to solve while achieving remarkably good accuracy for easy and medium complexity tasks. Following this observation, we show that by utilizing a hybrid inference strategy that tries a small model first and escalates to a large one only when needed, we can achieve up to 33% reduction in total GPU-hour costs and 16.4% on all tasks from SWE-bench Verified, without sacrificing accuracy. Strikingly, this simple cascade nearly matches an oracle-style task router, highlighting a practical path for deploying cost-aware coding agents in production without complex learned task routers and with minimal engineering complexity.
Subjects: Artificial Intelligence (cs.AI); Software Engineering (cs.SE)
Cite as: arXiv:2611.24324 [cs.AI]
  (or arXiv:2611.24324v1 [cs.AI] for this version)
  https://doi.org/10.48550/arXiv.2611.24324

arXiv-issued DOI via DataCite (pending registration)

Submission history

From: German Sviridov [view email]
[v1] Wed, 18 Nov 2026 12:26:42 UTC (180 KB)

来源:arXiv cs.AI · arxiv.org