MoE-SpAc：基于投机激活效用的异构边缘场景高效 MoE 推理框架
MoE-SpAc: Efficient MoE Inference Based on Speculative Activation Utility in Heterogeneous Edge Scenarios
AI 导读
论文提出 MoE-SpAc，将 Speculative Decoding 重新用作内存管理的前瞻传感器，以缓解 MoE 模型在边缘设备上的内存限制。框架包含投机效用估计器、通过在线整数优化动态划分计算的异构负载均衡器，以及统一预取与淘汰的异步执行引擎。在七个 benchmark 上，TPS 较 SOTA SD 基线提升 42%，较所有标准基线平均加速 4.04x，代码已开源。
正文 · 原文
Abstract:Mixture-of-Experts (MoE) models enable scalable performance but face severe memory constraints on edge devices. Existing offloading strategies struggle with I/O bottlenecks due to the dynamic, low-information nature of autoregressive expert activation. In this paper, we propose to repurpose Speculative Decoding (SD) not merely as a compute accelerator, but as an informative lookahead sensor for memory management, supported by our theoretical and empirical analyses. Hence, we introduce MoE-SpAc, an MoE inference framework that integrates a Speculative Utility Estimator to track expert demand, a Heterogeneous Workload Balancer to dynamically partition computation via online integer optimization, and an Asynchronous Execution Engine to unify the prefetching and eviction in the same utility space. Extensive experiments on seven benchmarks demonstrate that MoE-SpAc achieves a 42% improvement in TPS over the SOTA SD-based baseline, and an average 4.04x speedup over all standard baselines. Code is available at this https URL .
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computation and Language (cs.CL) |
| Cite as: | arXiv:2603.09983 [cs.LG] |
| (or arXiv:2603.09983v2 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2603.09983 arXiv-issued DOI via DataCite |
Submission history
From: Shuhuai Li [view email]
[v1]
Thu, 12 Feb 2026 04:28:00 UTC (3,135 KB)
[v2]
Fri, 23 Oct 2026 17:17:37 UTC (3,852 KB)
来源：arXiv cs.AI · arxiv.org