arXiv cs.AI· Jannatun Noor, Athar Noor Mohammad Rafee, Md. Sadman Haque, Sreezon Das Gupta, Atiar Osman, Md. Abu Obaida Zishan·· 3 小时前AI 评分39
云端 LLM：架构、编排与推理的系统综述
LLMs in Cloud: A Discussion on Architecture, Orchestration, and Inference
AI 导读
一篇按 PRISMA 2020 框架开展的跨层系统性综述，梳理云端大语言模型在 HBM 墙、KV cache 膨胀、无服务器冷启动等瓶颈下的部署挑战。
正文 · 原文
Abstract:The widespread deployment of foundational Large Language Models (LLM) across cloud datacenters marks a fundamental paradigm shift from stateless deep learning toward distributed, memory-intensive generative workloads. However, serving massive foundation models under stringent Service Level Objectives (SLO) exposes critical physical and system bottlenecks. These challenges include the persistent High Bandwidth Memory (HBM) wall, operational asymmetry between compute-bound prefilling and memory-bandwidth-bound decoding, explosive Key-Value (KV) cache expansion, prohibitive serverless cold starts, and hardware reliability vulnerabilities. Existing literature reviews remain layer-siloed, decoupling low-level accelerator virtualization from cluster orchestration and runtime inference, while largely overlooking defining advancements from 2024 to 2026. To address these limitations, this study presents a comprehensive, cross-layer systematic review guided by the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) 2020 framework. We synthesize a holistic taxonomy unifying hardware virtualization, cluster-level scheduling, and runtime inference optimizations, including iteration-level continuous batching, prefill-decoding disaggregation, and hierarchical KV cache offloading. Furthermore, we evaluate resilient cloud-native service architectures, examining scale-to-zero serverless paradigms, fault-tolerant checkpointing against silent data corruption, and hardware-enforced Trusted Execution Environments (TEEs) for multi-tenant isolation. Finally, we formalize the multi-dimensional Pareto trade-offs governing throughput, latency, monetary cost, and confidentiality, outlining a forward-looking research roadmap spanning Compute Express Link (CXL) memory pooling, in-memory computing, and autonomous multi-agent control planes to guide future cloud LLM infrastructure.
| Subjects: | Distributed, Parallel, and Cluster Computing (cs.DC); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.33475 [cs.DC] |
| (or arXiv:2610.33475v1 [cs.DC] for this version) | |
| https://doi.org/10.48550/arXiv.2610.33475 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Md Abu Obaida Zishan [view email]
[v1]
Sat, 24 Oct 2026 05:53:48 UTC (586 KB)
来源：arXiv cs.AI · arxiv.org