研究揭示 LLM decode 阶段功率上限的假象：锁 SM 时钟能省最多 32% 能耗
The Illusion of Power Capping in LLM Decode: A Phase-Aware Energy Characterisation Across Attention Architectures
AI 导读
一篇 arXiv 论文（arXiv:2605.11999，已被 PPAM 2026 接收）发现功率上限对主导 LLM 服务的自回归 decode 几乎无效。
推荐理由
原文用 H200 实测指出功率上限对 decode 几乎无效，并给出锁 SM 时钟能省最多 32% decode 能耗的可操作替代方案。
正文 · 原文
Abstract:Power capping is a common GPU energy lever in LLM serving, and it appears to work: the cap is set, the GPU reports drawing less than it, and the energy budget is met. We show the cap does little for the phase that dominates LLM serving: autoregressive decode. Across four attention paradigms (GQA, MLA, Gated DeltaNet, and Mamba2) on an NVIDIA H200 under vLLM eager execution, decode kernels wait on HBM and the GPU idles between kernel launches, so decode draws less than half of the GPU's 700W rating and throughput stays within a few percent under every cap we set. The cap engages only when a large batch runs a long generation. Locking the SM clock instead lowers power without slowing memory-bound decode and recovers up to 32% of decode energy at minimal throughput loss. We identify three architecture-dependent frequency-scaling classes and find that, once prefill is measured without prefix caching, the recurrent architectures pay no prefill penalty: their linear-time mixing undercuts quadratic attention on long prompts, so their efficient decode adds to the prefill saving and lowers total request energy from the first output token at large batch.
| Comments: | Accepted by PPAM 2026 (16th International Conference on Parallel Processing and Applied Mathematics) |
| Subjects: | Distributed, Parallel, and Cluster Computing (cs.DC); Artificial Intelligence (cs.AI); Machine Learning (cs.LG); Performance (cs.PF) |
| Cite as: | arXiv:2605.11999 [cs.DC] |
| (or arXiv:2605.11999v2 [cs.DC] for this version) | |
| https://doi.org/10.48550/arXiv.2605.11999 arXiv-issued DOI via DataCite |
Submission history
From: Bole Ma [view email]
[v1]
Tue, 12 May 2026 11:48:16 UTC (237 KB)
[v2]
Thu, 29 Oct 2026 15:43:01 UTC (158 KB)
来源：arXiv cs.AI · arxiv.org