CUDA-L2：用强化学习让矩阵乘法 CUDA 内核性能超越 cuBLAS
CUDA-L2: Surpassing cuBLAS Performance for Matrix Multiplication through Reinforcement Learning
AI 导读
CUDA-L2 结合大语言模型（LLM）与强化学习，以 CUDA 执行速度作为 RL 奖励，自动优化 HGEMM CUDA 内核，覆盖 1,000 种配置。离线模式下平均比 cuBLAS 快 +19.2%，比 cuBLASLt-AutoTuning 快 +11.4%；服务器模式下加速进一步提升至 +26.0% 和 +15.9%。项目代码已开源。
正文 · 原文
Abstract:In this paper, we propose CUDA-L2, a system that combines large language models (LLMs) and reinforcement learning (RL) to automatically optimize Half-precision General Matrix Multiply (HGEMM) CUDA kernels. Using CUDA execution speed as the RL reward, CUDA-L2 automatically optimizes HGEMM kernels across 1,000 configurations. CUDA-L2 systematically outperforms major matmul baselines to date, from the widely-used this http URL to state-of-the-art Nvidia's closed-source libraries, i.e., cuBLAS, cuBLASLt. In offline mode, where kernels are executed consecutively without time intervals, CUDA-L2 yields +22.0% over this http URL on average; +19.2% over cuBLAS using the optimal layout configuration (normal-normal NN and transposed-normal TN); +16.8% over cuBLASLt-heuristic, which queries cuBLASLt library and selects the algorithm based on the heuristic's suggestion; and +11.4% over the most competitive cuBLASLt-AutoTuning model, which selects the fastest algorithm from up to 100 candidates from cuBLASLt's suggestions. In server mode, where kernels are executed at random intervals simulating real-time inference, the speedups further increase to +28.7%, +26.0%, +22.4%, and +15.9% for this http URL, cuBLAS, cuBLASLt-heuristic, and cuBLASLt-AutoTuning respectively. CUDA-L2 shows that even the most performance-critical, heavily-optimized kernels like HGEMM can be improved through LLM-guided RL automation by systematically exploring configuration spaces at scales impractical for humans. Project and code can be found at this http URL
| Comments: | Project code: this http URL |
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2512.02551 [cs.LG] |
| (or arXiv:2512.02551v5 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2512.02551 arXiv-issued DOI via DataCite |
Submission history
From: Jiwei Li [view email]
[v1]
Tue, 2 Dec 2025 09:20:15 UTC (545 KB)
[v2]
Fri, 12 Dec 2025 00:47:40 UTC (545 KB)
[v3]
Mon, 13 Jul 2026 00:51:09 UTC (1,494 KB)
[v4]
Wed, 5 Aug 2026 18:31:31 UTC (1,494 KB)
[v5]
Wed, 11 Nov 2026 16:36:13 UTC (465 KB)
来源：arXiv cs.AI · arxiv.org