五个 LLM 在异构认知与生物医学知识图谱上的评测：图接地对多跳推理与幻觉抑制的作用
Benchmarking Large Language Models on Heterogeneous Cognitive and Biomedical Knowledge Graphs: The Role of Graph Grounding in Multi-Hop Reasoning and Hallucination Mitigation
AI 导读
一项 arXiv 研究对 GPT-5.1、Gemini 2.0 Pro、Llama 3、Qwen 和 BioPT 五个 LLM 在 DrugBank、NeuroSynth、Cognitive Atlas、Brain Map、GDPKG 五个知识图谱上进行基准评测，评估 KG 接地（graph-grounded）与无关条件下的多跳推理与事实准确性。
正文 · 原文
Abstract:Large language models (LLMs) are increasingly used in biomedical and cognitive neuroscience, but their ability to reason reliably over heterogeneous knowledge graphs (KGs) remains unclear. We benchmark five LLMs (GPT-5.1, Gemini 2.0 Pro, Llama 3, Qwen, and BioGPT) across five KGs: DrugBank, NeuroSynth, Cognitive Atlas, Brain Map, and GDPKG. We introduce a multitier query dataset to evaluate reasoning depth and factual accuracy under KG-aware (graph-grounded) and KG-agnostic conditions. Inter-annotator agreement was substantial, with Cohen's kappa of 0.781 (range: 0.753-0.807) and Fleiss' kappa of 0.77. Results show that general-purpose models, particularly GPT-5.1 and Gemini 2.0 Pro, generally outperform domain-specific models on multi-hop and high-level reasoning tasks, although performance declines as query complexity increases. KG grounding reduces hallucination rates across all evaluated models. Contrary to our initial hypothesis, domain-specific models do not consistently outperform general-purpose models on specialized tasks without graph grounding. These findings highlight the value of combining broad language-model capabilities with structured domain knowledge. We provide practical guidance for selecting LLM-KG integration strategies based on task complexity and computational constraints, and discuss hybrid neurosymbolic architectures for personalized medicine, cognitive disorder diagnosis, and explainable AI.
| Subjects: | Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Machine Learning (cs.LG) |
| Cite as: | arXiv:2611.08315 [cs.AI] |
| (or arXiv:2611.08315v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2611.08315 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Ali Sarabadani [view email]
[v1]
Thu, 5 Nov 2026 17:12:58 UTC (888 KB)
来源：arXiv cs.AI · arxiv.org