arXiv cs.AI· Liangwei Nathan Zheng, Wenhao Liang, Wei Emma Zhang, Lin Yue, Weitong Chen·· 5 小时前AI 评分43
混合专家（MoE）应对多模态学习挑战：一篇综述
Tackling Multimodal Learning Challenges with Mixture-of-Expert: A Survey
AI 导读
一篇被 IJCAI 2026 接收的综述系统梳理了 Mixture-of-Experts（MoE）如何解决多模态学习挑战，从三个视角展开：作为高效多模态引擎通过选择性专家激活解耦计算成本与参数增长、作为多模态表征学习器融合多专家知识、以及作为多模态适配器处理模态失衡与模态缺失等不完美数据场景。作者指出可解释路由、专家通信、模态整合和终身多模态学习等研究空白。
正文 · 原文
Abstract:Mixture-of-Experts (MoE) presents a naturally compatible and scalable framework for multimodal learning, demonstrating strong adaptability across diverse modalities and tasks. Despite its growing success, a comprehensive and systematic review on the MoE metho addressing multimodal challenges remains lacking. Existing surveys tend to evaluate either multimodal learning or MoE independently from method taxonomy, overlooking the unique interplay between them. This survey fills that gap by answering a central question: \textit{How does MoE effectively resolve multimodal challenges?} We approach this from three key perspectives: (1) \textbf{MoE as an Efficient Multimodal Engine:} enabling scalable multimodal modeling by decoupling computational cost from parameter growth and mitigating modality redundancy through selective expert activation; (2) \textbf{MoE as a Multimodal Representation Learner:} integrating complementary multi-opinion expert knowledge to enrich alignment and interaction representations; and (3) \textbf{MoE as a Multimodal Adapter:} providing a modular and flexible mechanism to model imperfect data scenarios such as modality imbalance and missing modality. Through our extensive literature review, we identify critical research gaps, including interpretable routing, expert communication, modality integration, and lifelong multimodal learning. We position this survey as a foundation for future research toward interpretable and sustainable multimodal Mixture-of-Experts system.
| Comments: | This survey has been accepted by IJCAI 2026. Results were released by 30 April 2026. This latest replacement is an extension version of survey to the previous version, which has been submitted to ACM Computing Survey for peer-review |
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2605.27431 [cs.LG] |
| (or arXiv:2605.27431v2 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2605.27431 arXiv-issued DOI via DataCite |
Submission history
From: Liangwei Zheng [view email]
[v1]
Fri, 22 May 2026 05:01:21 UTC (232 KB)
[v2]
Wed, 28 Oct 2026 05:31:01 UTC (547 KB)
来源：arXiv cs.AI · arxiv.org