MicroWorld：用多模态属性属性图弥合多模态大语言模型的微观领域鸿沟
MicroWorld: Empowering Multimodal Large Language Models to Bridge the Microscopic Domain Gap with Multimodal Attributed Property Graph
AI 导读
MicroWorld 框架通过多模态属性属性图（MAPG）与视觉检索路径为多模态大语言模型补充结构化知识，无需领域微调。该图包含约 106K 节点和 2.62M 条类型化边；在 MicroVQA 上将 Qwen3-VL-8B-Instruct 准确率从 48.8% 提升至 67.1%，超过零样本 GPT-5（59.4%），同一检索上下文可使 GPT-5 达到 71.5%。
正文 · 原文
Abstract:Multimodal large language models (MLLMs) show remarkable potential for scientific reasoning, yet their performance in specialized domains such as microscopy remains limited by the scarcity of domain-specific training data and the difficulty of encoding fine-grained expert knowledge into model parameters. To bridge the gap, we introduce MicroWorld, a framework for structured-knowledge augmentation with a complementary visual retrieval pathway, using a multimodal attributed property graph (MAPG) without domain-specific fine-tuning. MicroWorld extracts biomedical entities and relations via scispaCy or LLM-based triplet mining, aligns images and entities in a shared embedding space using Qwen3-VL-Embedding, and assembles a knowledge graph comprising approximately 106K nodes and 2.62M typed edges spanning eight relation categories. At inference time, a graph-augmented retrieval pipeline matches query entities to the MAPG and injects structured knowledge context into the MLLM prompt. On MicroVQA, MicroWorld raises Qwen3-VL-8B-Instruct accuracy from 48.8% to 67.1% (+18.3 percentage points; 37.5% relative), exceeding zero-shot GPT-5 (59.4%). The identical retrieved context raises GPT-5 to 71.5%, and GPT-5's own entity extraction reaches 73.3%. On MicroBench, 8B Single-Pass improves overall accuracy by 4.0 percentage points. The benefits are greatest for knowledge-intensive reasoning; fine-grained visual discrimination and the additional visual-pathway contribution remain limited. Code and accompanying resources: this https URL.
| Comments: | 31 pages, 14 figures |
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI) |
| MSC classes: | F.2.2, I.2.7 |
| Cite as: | arXiv:2605.10120 [cs.CV] |
| (or arXiv:2605.10120v2 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2605.10120 arXiv-issued DOI via DataCite |
Submission history
From: Manyu Li [view email]
[v1]
Mon, 11 May 2026 07:35:55 UTC (1,409 KB)
[v2]
Sun, 25 Oct 2026 09:12:01 UTC (2,756 KB)
来源：arXiv cs.AI · arxiv.org