BeQu 基准提出开放式知识评估范式，衡量 LLM 主动陈述的知识
Beyond Questions: Evaluating the Stated Knowledge of LLMs
AI 导读
一篇 arXiv 论文提出"开放知识评估"新范式，不再依赖预定义问题，而是用开放式提示（如"告诉我你所知的关于 M.L. King 的一切"）评估 LLM 主动陈述的知识，避免基准设计者的可得性偏差。
正文 · 原文
Abstract:Parametric knowledge in large language models (LLMs) is a cornerstone of their success, yet remains poorly understood. Existing knowledge benchmarks typically rely on predefined questions (e.g., "What is the birth date of M.L. King?"), evaluating only knowledge that benchmark designers explicitly choose to query, a problematic availability bias. In this paper, we introduce open knowledge evaluation, a new paradigm for benchmarking the stated knowledge of LLMs. Instead of asking narrow questions, it evaluates models on the knowledge they choose to surface in response to open-ended elicitation prompts (e.g., "Tell me everything you know about M.L. King"). This shifts the focus from predefined answer retrieval toward characterizing the knowledge models naturally express. We instantiate this paradigm with BeQu (Beyond Questions), a benchmark of 10,000 entities paired with reference corpora for statement verification. Using BeQu, we evaluate a broad range of language models and analyze the effects of reasoning effort, model scale, prompt format, and knowledge domain. Data and leaderboard are available on this work's GitHub repository and at ANONYMIZED.
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2605.26937 [cs.CL] |
| (or arXiv:2605.26937v4 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2605.26937 arXiv-issued DOI via DataCite |
Submission history
From: Luca Giordano [view email]
[v1]
Tue, 26 May 2026 12:29:18 UTC (3,278 KB)
[v2]
Mon, 10 Aug 2026 09:35:46 UTC (3,299 KB)
[v3]
Tue, 13 Oct 2026 09:41:02 UTC (3,227 KB)
[v4]
Tue, 27 Oct 2026 11:11:36 UTC (3,236 KB)
来源：arXiv cs.AI · arxiv.org