arXiv cs.AI· Wenliang Liang, Zhiyuan Ning, Haolin Chen, Yuanchun Zhou, Wenjuan Cui, Yi Du·· 4 小时前AI 评分37
ProSearch：实验科学多约束协议检索基准发布
ProSearch: Benchmarking Multi-Constraint Protocol Retrieval in Experimental Science
AI 导读
研究者推出 ProSearch 基准，用于评估实验科学中满足多约束条件的协议检索能力。基准包含覆盖 60 个实验领域的 1,515 条查询，每条含 3-5 个显式约束。测试 19 个检索器后，最佳模型平均 nDCG@10 仅 0.290，经重排提升至 0.474，联合约束绑定被识别为主要瓶颈。该工作已被 NeurIPS 2026 评估与数据集赛道接收。
正文 · 原文
Abstract:In AI for Science scenarios, experimental design and method reproduction require retrieving protocols that satisfy multiple interdependent constraints, yet existing benchmarks lack systematic evaluation of this capability. To address this gap, we introduce ProSearch, a benchmark for multi-constraint protocol retrieval in experimental science. ProSearch has three key features: (1) realism, with queries grounded in real experimental records and expert-defined templates; (2) complexity, with 60 experimental fields and their dependencies; and (3) diagnosticity, with 3-5 explicit constraints per query, including numerical and negation constraints. We construct ProSearch through LLM-based extraction, data-driven constraint selection, controlled query generation, and evidence-based relevance validation. The resulting benchmark contains 1,515 queries across six experimental science domains. We conduct extensive experiments on ProSearch with 19 representative retrievers and complementary analyses. The best-performing retriever achieves an average nDCG@10 of only 0.290, which improves to 0.474 with reranking. Structured diagnostic analysis shows that explicit constraint matching provides a strong ranking signal, while manual error analysis identifies joint constraint binding as a key bottleneck. ProSearch provides a testbed for systematically evaluating scientific retrieval systems.
| Comments: | Accepted to NeurIPS 2026 (Evaluations and Datasets Track). 48 pages, 4 figures. Wenliang Liang and Zhiyuan Ning contributed equally. Code: this https URL. Data: this https URL |
| Subjects: | Information Retrieval (cs.IR); Artificial Intelligence (cs.AI); Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.41396 [cs.IR] |
| (or arXiv:2610.41396v1 [cs.IR] for this version) | |
| https://doi.org/10.48550/arXiv.2610.41396 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Zhiyuan Ning [view email]
[v1]
Fri, 30 Oct 2026 08:06:24 UTC (245 KB)
来源：arXiv cs.AI · arxiv.org