OpenFrontiers：开放研究问题库与科学再发现基准发布
OpenFrontiers: A Collection of Open Research Problems and a Benchmark for Scientific Rediscovery
AI 导读
OpenFrontiers 发布，包含收录 12,151 个开放研究问题的跨学科语料库，以及基于近期已解决数学问题的回顾性基准：基准含 200 个问题（110 个于 2026 年解决、90 个仍开放），均经 Lean 4 形式化并经自动化探测和两次独立审查，另有按质量筛选的 HQ-50 子集。
正文 · 原文
Abstract:Large language models (LLMs) have shown increasing promise in research-level problem solving, yet a comprehensive collection of such problems is still missing, and evaluating them on research-level problems remains challenging. We introduce OpenFrontiers, a cross-disciplinary resource that combines a structured corpus of open research problems with a retrospective benchmark of recently resolved problems in mathematics. The corpus records problem statements, context, progress, and other information for 12,151 problems. The benchmark contains 200 problems, 110 resolved in 2026 and 90 still open, each with a Lean~4 statement checked by automated probes and two independent reviews; a subset HQ-50 further filters based on quality score, etc. Our evaluation protocol grades formal proofs with two independent checkers in an isolated sandbox under closed-book, frozen-retrieval, and live-web conditions. Evaluating seven LLM agents on the HQ-50 subset under a standardized budget, we find that closed-book rediscovery is rare, at most 7 of 40 resolved problems, whereas live-web success mostly reflects retrieval of published solutions. By linking a broad research-problem corpus to a formally graded evaluation subset with documented solutions, OpenFrontiers provides a foundation for measuring research-level rediscovery and studying progress toward solving currently open scientific problems. The dataset and benchmark could be found in this https URL.
| Subjects: | Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Machine Learning (cs.LG) |
| Cite as: | arXiv:2610.37739 [cs.AI] |
| (or arXiv:2610.37739v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.37739 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Jiarui Yao [view email]
[v1]
Tue, 27 Oct 2026 16:49:33 UTC (137 KB)
来源：arXiv cs.AI · arxiv.org