KAIROS：面向智能体推理的有状态、上下文感知功耗优化系统
KAIROS: Stateful, Context-Aware Power-Efficient Agentic Inference Serving
AI 导读
KAIROS 是一个面向智能体 AI 推理服务的上下文感知功耗优化系统，将智能体上下文作为一级控制信号，联合管理 GPU 频率、单实例并发与多实例请求放置。该系统指出智能体服务请求携带跨工具交替轮次演化的长期上下文，降低 GPU 频率可能引发内存抖动。在软件与数据工程智能体任务上，KAIROS 平均降低 27%（最高 39.8%）的功耗，同时满足性能目标。
正文 · 原文
Abstract:Power has become a central bottleneck for AI inference. This problem is becoming more urgent as agentic AI emerges as a major workload class, yet prior power-management techniques focus almost entirely on single-turn LLM serving. Our analysis shows that agentic serving behaves fundamentally differently: each request carries long-lived context that evolves across tool-interleaved turns, and lowering GPU frequency can push the system into a thrashing regime where memory pressure sharply worsens both performance and power efficiency. These observations show that power optimization for agentic serving requires rethinking.
We present KAIROS, a context-aware power optimization system for agentic AI serving. KAIROS uses agent context as a first-class control signal to jointly manage GPU frequency, per-instance concurrency, and multi-instance request placement. This enables KAIROS to save power when memory headroom exists while avoiding thrashing and preserving performance targets. At a high level, KAIROS tracks requests at agent granularity, adapts local control to context growth and agent progress, and routes agents across instances to jointly improve power efficiency and memory stability. Evaluated across diverse software and data engineering agentic tasks, KAIROS achieves an average of 27% (up to 39.8%) power reduction while meeting the performance targets.
| Subjects: | Distributed, Parallel, and Cluster Computing (cs.DC); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2604.16682 [cs.DC] |
| (or arXiv:2604.16682v2 [cs.DC] for this version) | |
| https://doi.org/10.48550/arXiv.2604.16682 arXiv-issued DOI via DataCite |
Submission history
From: Yichao Yuan [view email]
[v1]
Fri, 17 Apr 2026 20:39:25 UTC (1,785 KB)
[v2]
Tue, 10 Nov 2026 05:17:57 UTC (1,149 KB)
来源：arXiv cs.AI · arxiv.org