arXiv cs.AI· Yu Dong, Hong Li, Qiang Sun, Xu Zhou, Jinbiao Huo, Qixiu Cheng, Zhiyuan Liu·· 2 小时前AI 评分40
LM-SimBench：面向大语言模型训练模拟的统一框架、数据集与基准
LM-SimBench: A Unified Framework, Dataset, and Benchmark for Large Language Model Training Simulation
AI 导读
研究者提出 LM-SimBench，一个用于评测大语言模型训练模拟器的统一框架、数据集与基准。该工作在 Ascend NPU 集群上剖析来自 10 个稠密与 MoE 模型族的 23 种模型配置，生成包含 348 种可行配置的数据集，覆盖五个并行维度与激活重计算。对代表性训练模拟器的评测表明其可支持预测精度比较与误差诊断，GPU-NPU 匹配实验验证了跨平台适用性。
正文 · 原文
Abstract:Evaluating candidate parallelization strategies through distributed training runs makes strategy selection prohibitively expensive at scale. Performance simulation offers a lower cost alternative, yet controlled comparisons among existing simulators remain difficult because they differ in the training stages they model and the workloads on which they are evaluated. We introduce LM-SimBench, a unified framework, profiling dataset, and benchmark for evaluating LLM training simulators. The framework decomposes simulation into functional stages from model compilation to schedule level aggregation, connecting execution representations and cost estimators to measurable evaluation targets. Following this framework, we profile 23 model settings from 10 dense and Mixture-of-Experts (MoE) LLM families on an Ascend Neural Processing Unit (NPU) cluster. The resulting dataset contains 348 feasible configurations that jointly vary five parallelism dimensions and activation recomputation, with six views spanning configuration, rank, operator, event, and kernel granularity. Evaluations of representative training simulators demonstrate how LM-SimBench supports comparisons of prediction accuracy, assessment of configuration selection, and diagnosis of prediction errors. Matched GPU-NPU experiments validate the cross-platform applicability of the collection and evaluation protocol and demonstrate NPU-guided GPU configuration selection.
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2611.14285 [cs.AI] |
| (or arXiv:2611.14285v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2611.14285 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Yu Dong [view email]
[v1]
Tue, 10 Nov 2026 16:26:19 UTC (9,218 KB)
来源：arXiv cs.AI · arxiv.org