跳到正文
arXiv cs.AI· Zhihang Yi, Jian Zhao, Jiancheng Lv, Tao Wang·· 2 小时前AI 评分33

图表理解的多模态信息融合:MLLM 综述——演进、局限与认知增强

Multimodal Information Fusion for Chart Understanding: A Survey of MLLMs -- Evolution, Limitations, and Cognitive Enhancement

AI 导读

一篇发表于 Information Fusion 的综述系统梳理了多模态大语言模型(MLLM)在图表信息融合领域的进展。文章分析了图表中视觉与语言信息融合的核心挑战,提出涵盖规范与非规范基准的分类体系,并回顾了从经典深度学习到 MLLM 融合范式的方法演进。针对现有模型在感知与推理上的不足,综述指出高级对齐技术与强化学习认知增强是有前景的方向。

正文 · 原文

View PDF HTML (experimental)

Abstract:Chart understanding is a quintessential information fusion task, requiring the seamless integration of graphical and textual data to extract meaning. The advent of Multimodal Large Language Models (MLLMs) has revolutionized this domain, yet the landscape of MLLM-based chart analysis remains fragmented and lacks systematic organization. This survey provides a comprehensive roadmap of this nascent frontier by structuring the domain's core components. We begin by analyzing the fundamental challenges of fusing visual and linguistic information in charts. We then categorize downstream tasks and datasets, introducing a novel taxonomy of canonical and non-canonical benchmarks to highlight the field's expanding scope. Subsequently, we present a comprehensive evolution of methodologies, tracing the progression from classic deep learning techniques to state-of-the-art MLLM paradigms that leverage sophisticated fusion strategies. By critically examining the limitations of current models, particularly their perceptual and reasoning deficits, we identify promising future directions, including advanced alignment techniques and reinforcement learning for cognitive enhancement. This survey aims to equip researchers and practitioners with a structured understanding of how MLLMs are transforming chart information fusion and to catalyze progress toward more robust and reliable systems.
Subjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Machine Learning (cs.LG)
Cite as: arXiv:2602.10138 [cs.CV]
  (or arXiv:2602.10138v2 [cs.CV] for this version)
  https://doi.org/10.48550/arXiv.2602.10138

arXiv-issued DOI via DataCite

Journal reference: Information Fusion 138 (2027) 104700
Related DOI: https://doi.org/10.1016/j.inffus.2026.104700

DOI(s) linking to related resources

Submission history

From: Zhihang Yi [view email]
[v1] Sun, 8 Feb 2026 12:59:50 UTC (28,457 KB)
[v2] Wed, 18 Nov 2026 06:57:32 UTC (26,693 KB)

来源:arXiv cs.AI · arxiv.org