arXiv cs.AI· Abdul Hannan Azad, Irvin Wang, Jiwan Jiang, Chenchen Kuai, Bofeng Cao, Keshu Wu, Yuzhou Wang, Xiaopeng Li, Yang Zhou·· 2 小时前AI 评分30
GEMS-RAG：面向工程标准的图 grounding 多模态检索增强生成框架
GEMS-RAG: Graph-grounded Engineering Multimodal Standards Retrieval-Augmented Generation
AI 导读
GEMS-RAG 是一个针对工程标准的图 grounding 多模态检索增强生成框架，结合层级解析、细粒度视觉实体抽取、多模态知识图谱构建与图扩展混合检索，将图表分解为可关联文本条款的视觉实体。在 MUTCD 交通控制设备标准上的评测显示，其检索效果和问答性能优于纯文本、多模态和基于图的 RAG 基线，并可用于工程规范、技术手册等安全关键文档的推理。
正文 · 原文
Abstract:Engineering standards are challenging knowledge sources for large language models (LLMs) because they are hierarchical, multimodal, and densely connected through figures, tables, cross-references, and normative rules. Existing retrieval-augmented generation (RAG) methods often rely on unstructured text retrieval and struggle with questions requiring joint reasoning over textual provisions, visual specifications, and document structure. We propose GEMS-RAG (Graph-grounded Engineering Multimodal Standards Retrieval-Augmented Generation), a framework for grounded reasoning over engineering standards. GEMS-RAG combines hierarchical parsing, fine-grained visual entity extraction, multimodal knowledge graph construction, and graph-grounded hybrid retrieval. Instead of treating figures as monolithic images, it decomposes them into meaningful visual entities linked to textual provisions, engineering concepts, and regulatory rules. During inference, dense, sparse, and visual retrieval are combined with graph expansion to identify relevant evidence across text, figures, and document structure. We evaluate GEMS-RAG on the Manual on Uniform Traffic Control Devices (MUTCD), a large engineering standard containing egulations, technical drawings, and standardized traffic-control devices. Results show that graph-grounded multimodal retrieval improves retrieval effectiveness and question-answering performance over text-only, multimodal, and graph-based RAG baselines. Beyond MUTCD, GEMS-RAG offers a general framework for reasoning over engineering codes, technical manuals, regulatory documents, and scientific standards in safety-critical applications.
| Subjects: | Artificial Intelligence (cs.AI); Machine Learning (cs.LG) |
| Cite as: | arXiv:2610.33277 [cs.AI] |
| (or arXiv:2610.33277v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.33277 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Abdul Hannan Azad Ahanger [view email]
[v1]
Fri, 23 Oct 2026 20:41:44 UTC (6,972 KB)
来源：arXiv cs.AI · arxiv.org