跳到正文
arXiv cs.AI· Zihao Wang, Ye Tian, Tajana Rosing, Nalini Venkatasubramanian·· 2 小时前AI 评分44

LEDGE:面向异构边缘设备分布式 LLM 推理的高效模拟器

LEDGE: An Efficient Simulator for Distributed LLM Serving on Heterogeneous Edge Devices

AI 导读

研究者发布 LEDGE,一款无需执行张量运算即可快速评估异构边缘设备上分布式 LLM 推理的轻量模拟器。它支持非均匀张量与流水线并行,在 10 台 Jetson 和 Raspberry Pi 设备、17 种模型-精度组合上测试,Qwen 部署的请求级平均绝对百分比误差低于 3.6%,较物理推理快 135.3 倍;在四个边缘环境完成 26,773 次模拟仅需 83.8 分钟,源代码已公开。

正文 · 原文

View PDF HTML (experimental)

Abstract:Evaluating distributed large language model (LLM) serving on heterogeneous edge devices requires exploring diverse model partitions, device placements, and network conditions, making repeated physical experiments costly. An efficient simulator for edge LLM serving must support diverse, fine-grained placement configurations, capture runtime behavior, and accurately predict execution times while keeping simulation overhead low. We present LEDGE, a lightweight simulator for fast, fine-grained evaluation of distributed LLM serving on heterogeneous edge devices without executing tensor operations. LEDGE constructs event graphs supporting non-uniform tensor and pipeline parallelism, estimates execution costs using reusable performance models calibrated from end-to-end measurements, and simulates shared-link contention. We evaluate LEDGE on a heterogeneous testbed of 10 Jetson and Raspberry Pi devices, covering 17 model--precision combinations across 4 LLM families. On Qwen deployments, LEDGE achieves request-level mean absolute percentage errors below 3.6% and a 135.3 times aggregate speedup over physical inference. Performance model calibrated on one LLM family yield 7.44% zero-shot end-to-end latency MAPE across three other families. Further simulations show how shared-link contention affects placement decisions and network topology constrains scaling. Across four edge environments, LEDGE completes 26,773 simulation runs in 83.8 minutes, enabling efficient exploration of deployment strategies under diverse resource conditions and serving objectives. The source code is publicly available at this https URL.
Subjects: Distributed, Parallel, and Cluster Computing (cs.DC); Artificial Intelligence (cs.AI)
Cite as: arXiv:2611.24085 [cs.DC]
  (or arXiv:2611.24085v1 [cs.DC] for this version)
  https://doi.org/10.48550/arXiv.2611.24085

arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Zihao Wang [view email]
[v1] Wed, 18 Nov 2026 09:11:43 UTC (499 KB)

来源:arXiv cs.AI · arxiv.org