MECI:面向多模态实体链接的多元素协同交互框架
MECI: Multi-Element Collaborative Interaction for Multimodal Entity Linking
AI 导读
论文提出 MECI 框架,解决多模态实体链接(MEL)中的元素主导问题。框架包含多视图专家网络(配合 KL 引导路由)、层次化多模态交互模块,并利用大语言模型进行零样本重排序。在 WikiDiverse、RichpediaMEL 和 WikiMEL 三个基准上,Hits@1 分别提升 1.95%、7.30% 和 2.31%。
正文 · 原文
Abstract:Multimodal Entity Linking (MEL) aims to disambiguate mentions in multimodal contexts by grounding them to specific entities in a knowledge base. A pivotal challenge in MEL is capturing multi-level correspondences: the semantic consistency between mention-entity pairs and the complementary correlations across modalities. However, existing methods often suffer from element dominance due to their reliance on coupled interactions or coarse global aggregations. In response, we propose the Multi-Element Collaborative Interaction (MECI) framework. First, to capture multi-element mention-entity correspondences, we develop a Multi-view Experts Network that leverages a "divide-and-conquer" strategy for decoupled feature learning to mitigate element dominance, supported by a KL-guided routing mechanism that governs expert specialization and collaboration. Furthermore, to model cross-modal complementary correlations, we propose a Hierarchical Multimodal Interaction Module, where a dynamic modality-aware weighting network refines interactions across hierarchical semantic levels, thereby integrating multi-granular evidence to counteract element dominance. Finally, we incorporate a generative semantic refinement stage that utilizes large language models for zero-shot re-ranking. Extensive experiments on WikiDiverse, RichpediaMEL, and WikiMEL show that MECI consistently outperforms state-of-the-art baselines, improving Hits@1 by 1.95%, 7.30%, and 2.31%, respectively.
| Comments: | 11 pages, 8 figures |
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2611.24522 [cs.CL] |
| (or arXiv:2611.24522v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2611.24522 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Jie Peng [view email]
[v1]
Wed, 18 Nov 2026 14:39:07 UTC (5,275 KB)
来源:arXiv cs.AI · arxiv.org