arXiv cs.AI· Yavuz Yetim, Bor-Yiing Su, Zhe Wang, Arthur Liu, Pranav Sharma, Ying Wang, Randy Shuai, Luke Pfister, Ruichao Xiao, Bill Zhao, Zihao Liu, Liyuan Li, Jiangyuan Li, Gaoxiang Liu, Nathan Yan, Geonhwa Jeong, Jatin Chhugani, Elias Ellison, Uladzimir Pashkevich·· 4 小时前AI 评分43
用 Microscaling 格式为推荐与排序模型推进量化(arXiv 论文)
Advancing Quantization for R&R Models with Microscaling Formats
AI 导读
一篇 arXiv 论文在 AMD MI350X 上评测了 MXFP8 和 6-bit MXFP6 格式用于工业级推荐与排序(R&R)模型推理,这类广告模型要求模型质量相对 FP16/FP32 基线保持在 0.01% Normalized Entropy 余量内。
正文 · 原文
Authors:Yavuz Yetim, Bor-Yiing Su, Zhe Wang, Arthur Liu, Pranav Sharma, Ying Wang, Randy Shuai, Luke Pfister, Ruichao Xiao, Bill Zhao, Zihao Liu, Liyuan Li, Jiangyuan Li, Gaoxiang Liu, Nathan Yan, Geonhwa Jeong, Jatin Chhugani, Elias Ellison, Uladzimir Pashkevich
Abstract:Recommendation and Ranking (R&R) models, particularly in digital advertising, require massive inference throughput but exhibit extreme sensitivity to precision loss. Unlike Large Language Models (LLMs) that frequently tolerate quantization-induced accuracy drops of 1% or more, production-grade R&R models enforce a stringent launch bar: model quality must remain within a 0.01% Normalized Entropy (NE) margin relative to the baseline, which commonly uses FP16 or FP32. This tight constraint has historically hindered the adoption of aggressive quantization in this domain. In this paper, we present an empirical evaluation of both MXFP8 and the 6-bit MXFP6 format for industrial-scale R&R inference on the AMD MI350X architecture.
To the best of our knowledge, this is the first paper, across R&R and LLM domains, that reports significant speedups from native MXFP6 GEMMs, which quantize both weights and activations and compute directly on 6-bit elements, without sacrificing model quality. We demonstrate that we can selectively apply this 6-bit format to maximize overall QPS gain without breaching the strict NE parity threshold. We further show that MXFP8 is a viable alternative to row-wise FP8: the two formats have nearly the same accuracy on our models, so the faster kernel can be chosen for each GEMM shape. On two production ads ranking models, our implementation shows measured GEMM kernel speedups over FP16 of 1.33--1.55x for MXFP8 and 1.73--1.99x for MXFP6 across all supported linear layers, and up to 2.90x for MXFP6 on individual layers, alongside end-to-end (E2E) inference throughput gains from selective MXFP6 of 1.5--5.3% for Model A and 7.6--14.3% for Model B over their FP8 configurations.
| Subjects: | Artificial Intelligence (cs.AI) |
| ACM classes: | I.2.6; G.1.0; C.4 |
| Cite as: | arXiv:2611.17501 [cs.AI] |
| (or arXiv:2611.17501v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2611.17501 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Bor-Yiing Su [view email]
[v1]
Thu, 12 Nov 2026 20:46:31 UTC (57 KB)
来源:arXiv cs.AI · arxiv.org