FrontierOR 基准发布：评测 LLM 设计大规模优化算法的能力
FrontierOR: Benchmarking LLMs' Capacity for Efficient Algorithm Design in Large-Scale Optimization
研究团队发布 FrontierOR，一个基于顶级运筹学期刊论文构建的 180 任务基准，用于评测 LLM 生成的大规模优化算法效率。结果显示当前模型难以从可执行的建模走向高效算法，最强 LLM 在 Full-set 上仅 39% 任务实例达成目标差距不超过 1% 的可行解；在 Hard 子集上，测试时自进化可将最强单次生成模型的成功率从 51% 提升到 78%。
Authors:Minwei Kong, Chonghe Jiang, Ao Qu, Wenbin Ouyang, Zhaoming Zeng, Xiaotong Guo, Zhekai Li, Junyi Li, Yi Fan, Xinshou Zheng, Xi Jing, Yikai Zhang, Zhiwei Liang, Seonghoo Kim, Runqing Yang, Zijian Zhou, Sirui Li, Han Zheng, Wangyang Ying, Ou Zheng, Chonghuan Wang, Jinglong Zhao, Hanzhang Qin, Cathy Wu, Paul Pu Liang, Jinhua Zhao, Hai Wang
Abstract:Large language models (LLMs) are increasingly used for optimization modeling and solver-code generation, yet practical operations research and optimization problems often require a harder capability: designing scalable algorithms that exploit problem structure and outperform direct formulation-and-solve baselines. Existing benchmarks are limited to small or simplified examples far below real-world scale and complexity. We introduce FrontierOR, the first literature-grounded benchmark targeting LLM-generated algorithm efficiency on realistic large-scale optimization problems. FrontierOR includes 180 diverse tasks, each derived from a paper published in a top-tier operations research venue, with standardized instances and a hidden, expert-verified evaluation suite. We evaluate recent LLMs' one-shot algorithm generation and improvements achieved through several self-evolving frameworks. The results show that current models still struggle to move from executable formulations to efficient optimization algorithms. Even the strongest LLM succeeds on only 39% of Full-set task instances, where success is defined as returning a feasible solution with an objective gap of at most 1% relative to Gurobi within its runtime. On the Hard subset, test-time evolution raises this proportion from 51% for the strongest one-shot model to 78%. FrontierOR provides a continuously maintained testbed for evaluating LLM-designed optimization algorithms under a shared protocol and controlled computational settings. Code and data are publicly released at this https URL.
| Comments: | Accepted as a poster at NeurIPS 2026 (Evaluations and Datasets Track) |
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2605.25246 [cs.AI] |
| (or arXiv:2605.25246v4 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2605.25246 arXiv-issued DOI via DataCite |
Submission history
From: Minwei Kong [view email]
[v1]
Sun, 24 May 2026 20:10:42 UTC (1,785 KB)
[v2]
Tue, 26 May 2026 13:18:52 UTC (1,785 KB)
[v3]
Sat, 30 May 2026 22:51:15 UTC (1,785 KB)
[v4]
Wed, 11 Nov 2026 22:12:04 UTC (1,654 KB)
来源：arXiv cs.AI · arxiv.org