用恢复的证据图约束视觉语言模型实现 P&ID 图纸的接地推理
Grounded and Faithful P&ID Reasoning: Constraining Vision-Language Models with Recovered Evidence Graphs
AI 导读
研究者提出 Graph-Grounded Harness：先从 P&ID 工艺管道仪表图中恢复由符号、工艺连接和标签构成的显式图，再让模型只能通过七个只读算子查询该图来作答，拓扑结论须引用查询结果。
正文 · 原文
Abstract:Piping and Instrumentation Diagrams (P&IDs) are the authoritative maps of process plants: isolation, maintenance, and HAZOP decisions depend on what connects to what. Vision-language models describe these sheets fluently, yet they often invent or miss process connections---and an invented or missed link can reverse an isolation or reachability call, so a plant decision cannot trust a fluent answer that was never checked against the linework. We instead recover an explicit graph of the drawing---its symbols, the process connections between them, and the tags that name them---and then require the model to answer only by querying that graph through seven read-only operators, so a topology claim is returned only when it cites the query results that support it. On TopoPID-VQA, a new suite of 3000 topology questions over these sheets, Graph-Grounded Harness (Ours) raises exact match accuracy from 36.7--41.3% under image-only prompting to 74.3--76.0% for Qwen3-VL-4B, Qwen3-VL-8B, and Gemma-4-E4B. It does so on an imperfect substrate: on Digitize-PID dataset the recovered graph scores F1 0.742 on exact process connections, and 0.801 once symbols and tags are pooled in. The residual errors track that gap---grounding pays off where the recovered graph is right, and perception error still breaks topology questions where it is not.
| Comments: | Accepted to the NeurIPS 2026 VLM4RWD Workshop. Changes in v2: (1) added answer-contract ablation results (Table 2), as suggested by reviewers; (2) added fine-tuned VLM results; (3) added the harness system prompts (Appendix A.9) and appendix sections describing the answer-contract ablation (A.10) and the fine-tuning experiments (A.11) |
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2609.05880 [cs.LG] |
| (or arXiv:2609.05880v2 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2609.05880 arXiv-issued DOI via DataCite |
Submission history
From: Prathamesh Gadekar [view email]
[v1]
Sat, 5 Sep 2026 05:02:34 UTC (3,794 KB)
[v2]
Thu, 29 Oct 2026 07:48:32 UTC (7,426 KB)
来源：arXiv cs.AI · arxiv.org