本体作为 LLM Agent 知识图谱检查器能查出什么？人工、LLM 生成与数据驱动本体对比
What Does an Ontology Catch? Curated, LLM-Generated and Data-Derived Ontologies as Checkers of LLM Agents over Knowledge Graphs
AI 导读
arXiv 论文研究了基于知识图谱本体的检查器能否可靠判定 LLM Agent 的答案，对比人工整理、LLM 生成和数据挖掘三种本体，在公开图谱及约 $10^{9}$ 三元组的私有图谱上测试。结果显示，在 UniProt 上人工本体错误标记的正确答案比另两种更多，其标记精度仅略高于随机标记。
正文 · 原文
Abstract:LLM agents that answer questions over Web knowledge graphs are increasingly wrapped in checks derived from the graph's ontology, on the assumption that a formal schema makes a reliable judge. Yet a checker can be wrong as well: it can miss errors, and it can flag correct answers and push the agent into "fixing" them. Whether a curated ontology is worth its cost as such a judge, and how it compares with an ontology an LLM writes or a schema mined from the data, is unknown. We make the checker itself the object of study. Across curated, LLM-generated and data-derived ontologies, used as prompt context, as a checker, or both, and across open-weight and proprietary agents on public graphs and a private graph of nearly $10^{9}$ triples, we account for what ontology-derived checks catch and what they wrongly flag. We show how each source trades caught errors against false alarms, how a curated schema used as published can contradict its own data, which errors structural checks miss, and whether combining sources or giving the agent the checker's specific feedback pays off. On UniProt, the curated ontology flags more correct answers than either an ontology derived from the data or one an LLM derives from a summary of the data with masked identifiers. On real agent answers, its flags are only marginally more precise than random flags.
| Comments: | 13 pages, 2 figures. Under review |
| Subjects: | Artificial Intelligence (cs.AI); Databases (cs.DB) |
| Cite as: | arXiv:2610.37809 [cs.AI] |
| (or arXiv:2610.37809v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.37809 arXiv-issued DOI via DataCite |
Submission history
From: Michal Orzechowski [view email]
[v1]
Tue, 27 Oct 2026 18:27:45 UTC (54 KB)
来源：arXiv cs.AI · arxiv.org