arXiv cs.AI· Nan Huang, Mario Tapia-Pacheco, Kun Zhou, Yiming Huang, Kevin Jos\'e Barrientos D\'iaz, Tiffany Amariuta, Jingbo Shang·· 4 小时前AI 评分47
DISCERN 基准：AI 智能体能像科学家一样引导科学发现吗？
DISCERN: Can AI Agents Work Like Scientists and Guide Discovery?
AI 导读
arXiv 论文提出 DISCERN 基准，基于真实公开数据集从数据完整性、分析验证、假设生成三个层级评测自动化科研智能体。覆盖 203 个任务、8 个生命科学赛道和 8 个模型，智能体仅在 60.8% 的 Level 1、34.2% 的 Level 2 和 0.6% 的 Level 3 评测中拿到满分。结果表明当前智能体尚不能可靠地自主开展分析与科学发现，更适合有监督的科研辅助角色。
正文 · 原文
Abstract:Reliable automated research requires agents to vet data, verify analyses, and generate hypotheses grounded in trustworthy evidence, potentially reducing routine scientific workload while allowing scientists to focus on interpretation and discovery. Existing benchmarks often only assess analytical task completion or hypothesis generation separately rather than testing whether reliable evidence supports valid and novel claims. We introduce DISCERN (Data Integrity and Scientific Capability: Evidence, Reasoning, and Novelty), a controlled benchmark on real, publicly available datasets that evaluates three key levels of an automated research workflow. The first two levels test data integrity and analysis verification under confounds and tool traps, while the third tests hypothesis generation and revision under adversarial review, including counterfactual cases in which evidence consistent with real data and documented scientific phenomena conflicts with established expectations, motivating alternative explanations and testable hypotheses. Across 203 tasks, eight life-science tracks, and eight models, DISCERN shows that strong aggregate performance can mask level-specific weaknesses. Agents earn perfect scores in only 60.8% of Level 1, 34.2% of Level 2, and 0.6% of Level 3 evaluations, with penalties attributed to rejection of sound data, failure to carry recognized limitations into conclusions, and wide variation in hypothesis production. Cross-track rankings by token and code use are substantially more stable than rankings by evidence judgment, suggesting greater consistency in computational effort than in evidence-based reasoning. These profiles identify opportunities for supervised scientific assistance, but current agents do not yet demonstrate reliable autonomous analysis or discovery. Code and data: this https URL
| Comments: | Project website: this https URL. Updated to include the project website and planned life-science verification evaluation |
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2609.33357 [cs.AI] |
| (or arXiv:2609.33357v2 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2609.33357 arXiv-issued DOI via DataCite |
Submission history
From: Mario Tapia-Pacheco [view email]
[v1]
Sun, 27 Sep 2026 08:29:32 UTC (1,564 KB)
[v2]
Mon, 26 Oct 2026 05:50:43 UTC (1,564 KB)
来源：arXiv cs.AI · arxiv.org