跳到正文
arXiv cs.AI· Jie Peng, Yongxue Shan, Yongfu Zha, Cui Miao, Xiaodong Wang·· 1 小时前AI 评分32

NaviGATE:面向多模态知识图谱推理的噪声感知双视角生成框架

NaviGATE: Noise-Aware Dual-View Generation for Multimodal Knowledge Graph Reasoning

AI 导读

研究者提出 NaviGATE,一个面向多模态知识图谱推理(MKGR)的噪声感知生成框架。该方法先对齐图结构表示与文本语义,再用多模态实体摘要与注入的结构 token 微调多模态语言模型,并在双视角指令微调中通过对比 logit 锐化强化可靠检索候选、用参数一致性蒸馏抑制误导候选,推理时以结构约束解码限制生成有效实体。

正文 · 原文

View PDF HTML (experimental)

Abstract:Multimodal knowledge graph reasoning (MKGR) aims to infer missing facts by jointly exploiting relational structures, textual descriptions, and visual evidence. Existing embedding-based methods effectively capture graph topology, but their fixed multimodal representations are less effective for query-specific fine-grained semantic reasoning, while generation-based methods are vulnerable to noisy retrieval and may produce invalid entities. To address these challenges, we propose NaviGATE, a noise-aware generative framework for MKGR. NaviGATE first aligns graph-derived structural representations with textual semantics, then performs structure-aware internal knowledge activation by fine-tuning the multimodal language model with multimodal entity summaries and injected structural tokens. In the noise-aware dual-view instruction tuning stage, NaviGATE jointly optimizes retrieval-free and retrieval-aware views: reliable retrieved candidates are reinforced through contrastive logit sharpening, while misleading candidates are suppressed through parametric consistency distillation. In inference, structure-constrained decoding restricts generation to valid entities in the target knowledge graph. Experiments on standard benchmarks show that NaviGATE consistently outperforms strong unimodal, multimodal, and generation-based baselines, and remains robust under noisy retrieval scenarios.
Comments: 9 pages, 4 figures
Subjects: Artificial Intelligence (cs.AI)
Cite as: arXiv:2611.24552 [cs.AI]
  (or arXiv:2611.24552v1 [cs.AI] for this version)
  https://doi.org/10.48550/arXiv.2611.24552

arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Jie Peng [view email]
[v1] Wed, 18 Nov 2026 14:53:42 UTC (552 KB)

来源:arXiv cs.AI · arxiv.org