跳到正文
arXiv cs.AI· Bing Wang, Mingyang Wang, Yuanhang Mi, Yue Xu, Kaibo Su, Li'ang Xiao, Chunguo Wu, Changchun Li, Ximing Li, Kun-Feng Qiu, Bo Yang·· 4 小时前AI 评分33

MineralExp:面向真实矿产勘查实验设计的 LLM 智能体基准

MineralExp: Benchmarking LLM Agents for Experimental Design in Real-World Mineral Exploration

AI 导读

研究者发布 MineralExp 基准,将地球化学实验设计转化为基于假设的实验选择多标签分类问题,用于评估 LLM 智能体的上游实验设计能力。基准基于已发表矿床文献构建并经领域专家核验,包含 277 个案例和 1,572 份参考实验方案,抽象出覆盖源区、搬运、沉积、改造与保存五类成矿系统过程的 19 种实验类型。结果显示现有智能体召回率高但精确率低,倾向于提出超出验证假设所需的实验。

正文 · 原文

Authors:Bing Wang, Mingyang Wang, Yuanhang Mi, Yue Xu, Kaibo Su, Li'ang Xiao, Chunguo Wu, Changchun Li, Ximing Li, Kun-Feng Qiu, Bo Yang

View PDF HTML (experimental)

Abstract:In mineral exploration, verifying geological hypotheses relies on designing a variety of geochemical experiments. While large language model (LLM) agents increasingly automate downstream geoscientific data analysis, a crucial upstream capability remains largely unexplored: automated geochemical experimental design, i.e., deciding which experiments to conduct before committing to costly laboratory work. Evaluating this capability of LLM agents is difficult for two reasons. First, geochemical experiments exhibit high methodological diversity and subjectivity: to test the same hypothesis, different geologists may choose different minerals and instruments, so no single protocol can serve as the reference answer. Second, open-ended experimental plans resist representation as structured labels, which makes objective scoring difficult. To address these challenges, we introduce MineralExp, a benchmark that reformulates geochemical experimental design as a hypothesis-based experiment selection problem, i.e., a multi-label classification problem. We abstract concrete instrument names and procedural details into a shared taxonomy of 19 experimental types across five mineral-system processes: Source, Transport, Deposition, Modification, and Preservation. This taxonomy yields deterministic multi-label targets while accommodating diverse experimental protocols. Constructed from published ore-deposit literature and verified by domain experts, MineralExp comprises 277 cases and 1,572 reference experimental plans. Given a geological hypothesis and initial context, an LLM agent selects the relevant tasks and generates concrete experimental plans, which are scored by multi-label classification metrics and an LLM-as-a-Judge metric. The experimental results show that current agents generally achieve high recall but low precision, tending to propose more experiments than are necessary to test the hypothesis.
Comments: 16 pages, 6 figures
Subjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Computational Engineering, Finance, and Science (cs.CE)
Cite as: arXiv:2611.17931 [cs.CL]
  (or arXiv:2611.17931v1 [cs.CL] for this version)
  https://doi.org/10.48550/arXiv.2611.17931

arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Bing Wang [view email]
[v1] Fri, 13 Nov 2026 09:10:58 UTC (1,792 KB)

来源:arXiv cs.AI · arxiv.org