跳到正文
arXiv cs.AI· Yibang Li, Bihari Lal Pandey, Ravi Sah, Andi Han, Cyrus Mostajeran, Pratik Jawanpuria, Bamdev Mishra·· 2 小时前AI 评分38

iMuon:黎曼矩阵流形上的内在谱优化方法

iMuon: Intrinsic Spectral Optimization on Riemannian Matrix Manifolds

AI 导读

iMuon 是在 NeurIPS 2026 发表的优化框架,将 Muon 等范数约束优化器推广到流形参数。通过黎曼度量把酉不变欧氏范数提升为切空间内在范数,iMuon 在固定秩、SPD、Stiefel 和 Grassmann 流形上给出谱范数、Frobenius 范数、核范数下的闭式更新,收敛常数仅依赖流形维度(固定秩情形仅依赖秩)。

正文 · 原文

View PDF HTML (experimental)

Abstract:Muon and related norm-constrained optimizers, now central to large-scale learning, compute updates via a linear maximization oracle (LMO) over a matrix-norm ball in Euclidean space. However, they do not generalize cleanly to manifold-valued parameters such as low-rank factorizations, orthogonality constraints, or symmetric positive-definite (SPD) matrices. Naively restricting the Muon LMO to the tangent space (i) breaks quotient symmetries and (ii) couples the tangent-space constraint with an ambient norm bound, which obstructs closed-form solutions. We resolve both issues with a key observation: every Riemannian metric canonically lifts a unitarily invariant Euclidean norm to an intrinsic norm on each tangent space. The resulting intrinsic LMO is symmetry preserving and, under a structural condition on the scaled tangent space, reduces to a standard Euclidean LMO. Building on this, we introduce intrinsic Muon (iMuon), a unified framework with closed-form updates on the fixed-rank, SPD, Stiefel, and Grassmann manifolds for various unitarily invariant norms, including the spectral, Frobenius, and nuclear norms. We establish deterministic and stochastic convergence rates of iMuon with constants that depend only on the manifold dimension. Notably, on the fixed-rank manifold, the constant depends only on the rank, which makes the rate independent of factor conditioning and removes the runtime factor rescaling required by prior work. Experiments on LoRA fine-tuning of LLMs, image classification, and subspace learning illustrate the efficacy of the proposed approach.
Comments: Code: this https URL Accepted at NeurIPS 2026. Updated to match the camera-ready version
Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
Cite as: arXiv:2605.09238 [cs.LG]
  (or arXiv:2605.09238v2 [cs.LG] for this version)
  https://doi.org/10.48550/arXiv.2605.09238

arXiv-issued DOI via DataCite

Submission history

From: Yibang Li Nanyang technological university [view email]
[v1] Sun, 10 May 2026 00:39:13 UTC (356 KB)
[v2] Mon, 16 Nov 2026 23:41:40 UTC (272 KB)

来源:arXiv cs.AI · arxiv.org