Benchmark Radar：AI 评测基准的活数据库与搜索引擎
Benchmark Radar: A Living Database and Search Engine for AI Benchmarks and Evaluation
Benchmark Radar 是一个用于检索和发现 AI 评测基准的活数据库与搜索引擎，覆盖 LLM 评测、智能体与工具使用、编程、推理、安全和领域评测。其每日发现接入 37 个来源，目录含 1,283 条来源记录，并在 790 条记录上累积 12,916 条数值观测。项目提供网页仪表盘（含排行榜、Pareto 前沿视图、饱和度与趋势视图）、每日 feeds、CLI 离线查询及可复现分析。
Authors:Koutian Wu, Wenjun Zhang, Junjie Zhou, Pengqian Han, Qiming Bao, Ergan Shang, Jiayu Wang, Junkai Wang, Wanghan Xu, Songyuanyi Lu, Yixuan Li, Rudy Mizrahi Celekli, Lazizbek Ravshanov, Yaoliang Bian, Xudong Han, Lin Shi
Abstract:Benchmark researchers and developers of large language models (LLMs) and other AI systems need to find relevant evaluations, locate their benchmark datasets and code, and understand the settings behind reported scores. We present Benchmark Radar, a living database and search engine for retrieval and discovery of AI benchmarks, covering LLM evaluation, agentic and tool-use benchmarks, coding, reasoning, safety, and domain-specific evaluations. The system combines daily discovery of benchmark papers, repositories, datasets, and releases with a searchable benchmark catalog, mentions in model cards and technical reports, and score histories. It retains source identities and citations so readers can inspect candidate benchmarks and their evaluation evidence. Daily discovery draws on 37 sources: 13 direct connectors and 24 first-party research and engineering feeds. The catalog contains 1,283 source records drawn from 4 benchmark catalogs and 12,916 numeric observations on 790 records. We describe collection and retrieval, audit the full catalog, and examine benchmark saturation, adoption trends, and the limits of score comparisons. A worked example walks through a complete prior-art search, showing how to query the catalog and inspect benchmark evidence when designing a new evaluation. We release the web dashboard with a benchmark leaderboard, a Pareto frontier view of score against measured use, saturation and trend views, daily feeds, downloadable evidence, a command-line interface (CLI) for offline queries, and reproducible analysis.
| Comments: | Code: this https URL, Project site: this https URL |
| Subjects: | Artificial Intelligence (cs.AI); Information Retrieval (cs.IR) |
| Cite as: | arXiv:2609.11115 [cs.AI] |
| (or arXiv:2609.11115v4 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2609.11115 arXiv-issued DOI via DataCite |
Submission history
From: Koutian Wu [view email]
[v1]
Thu, 10 Sep 2026 05:47:22 UTC (4,314 KB)
[v2]
Sun, 13 Sep 2026 07:02:41 UTC (4,315 KB)
[v3]
Tue, 22 Sep 2026 03:36:21 UTC (4,316 KB)
[v4]
Tue, 27 Oct 2026 05:50:03 UTC (4,050 KB)
来源：arXiv cs.AI · arxiv.org