arXiv cs.AI· Juan Parra-Ullauri, Anthony Chiu, Hari Madhukumar, Zhaozhou Wu, Anderson Bravalheri, Alex Healing, Andy Corston-Petrie, Dimitra Simeonidou, Nektaria Efthymiou·· 3 小时前AI 评分42
InferSlice：面向电信广域网 SLO 驱动 LLM 推理的参考架构
InferSlice: A Reference Architecture for SLO-driven LLM Inference over a Telco Wide-Area Network
AI 导读
BT 联合布里斯托大学与 NVIDIA 提出 InferSlice，一套在电信基础设施上实现 SLO 驱动 LLM 推理的参考架构，集成网络切片、GPU 分区、弹性服务与可观测性，并在英国国家级广域网 JOINER 上实现。
正文 · 原文
Abstract:LLM inference is increasingly deployed as a core service for enterprise and interactive applications, but sovereignty, compliance, latency and resource-locality requirements can limit the suitability of centralised cloud deployments. Telco operators offer geographically distributed compute and network infrastructure for addressing these constraints. However, existing LLM serving frameworks are designed for tightly coupled data centres and provide limited support for coordinating geographically distributed and heterogeneous resources. We present InferSlice, a reference architecture for Service-Level Objective (SLO)-driven LLM inference over a telco infrastructure, integrating network slicing, GPU partitioning, elastic serving and observability. Led by BT in collaboration with the University of Bristol and NVIDIA, and realised on the JOINER UK national-scale wide-area network (WAN), InferSlice is evaluated on service differentiation across tiers, elastic scale-out, client-edge bandwidth as a bottleneck, and the reach of tensor, pipeline, data parallelism, and prefill/decode disaggregation over the WAN. Our results show that, on a general-purpose telco network, tiers yield distinct latency-throughput regimes, and weight streaming shortens scale-out time. Replication is placement-insensitive but memory-bound; pipeline parallelism tolerates near- and mid-cross-site hops, while tensor parallelism and disaggregation fail beyond a single subnet hop. We share practical insights from operating SLO-driven inference on a telco network, lessons learned and open questions, and hope to encourage further research on distributed and heterogeneous LLM inference systems.
| Subjects: | Distributed, Parallel, and Cluster Computing (cs.DC); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.32638 [cs.DC] |
| (or arXiv:2610.32638v1 [cs.DC] for this version) | |
| https://doi.org/10.48550/arXiv.2610.32638 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Juan Marcelo Parra [view email]
[v1]
Fri, 23 Oct 2026 13:51:02 UTC (8,589 KB)
来源：arXiv cs.AI · arxiv.org