WeightBench:将大语言模型作为权重空间程序员来评测的基准
WeightBench: Evaluating Language Models as Weight-Space Programmers
AI 导读
一篇 arXiv 论文提出 WeightBench,用覆盖读权重、修权重、写权重、识别不可行编辑和无数据压缩五条赛道的 42 项 WSP 任务评测大语言模型直接操作神经网络参数的能力,被测网络从 2K 参数控制策略到 7B 参数语言模型。
正文 · 原文
Abstract:Recent work shows that large language models (LLMs) can operate directly on the parameters of other neural networks. They can read what a network was trained to do, fix a network that fails, and write working networks from scratch, all without gradient descent. This capability has been termed Weight-Space Programming (WSP). Exploring this new paradigm systematically requires a robust benchmark. We introduce WEIGHTBENCH, a benchmark that evaluates LLMs of increasing capability on 42 WSP tasks across five tracks: reading, fixing and writing weights, recognizing edits that are impossible, and data-free compression. The networks whose weights the LLMs read and edit range from 2K-parameter control policies to 7B-parameter language models. Experimental benchmark results show that frontier agents can already read small networks and write working control policies without ever running them, but cannot yet patch language models, compress them without data, or tell an impossible edit from a hard one. On all 42 tasks, Claude Opus 5.5 reaches a macro score of 0.535 (Sonnet 5.5 0.466, Haiku 5.5 0.402, DeepSeek v4-flash 0.215); on the 38 tasks it was run on, GPT-6.1-sol reaches 0.568 against Opus's 0.546. Code, the task registry, our OS-level sandbox harness, and the live leaderboard are publicly available at this https URL.
| Comments: | 9 pages, 1 figure, 1 table. Code and leaderboard available at this https URL |
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2611.18553 [cs.LG] |
| (or arXiv:2611.18553v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2611.18553 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Aayush Marasini [view email]
[v1]
Fri, 13 Nov 2026 18:58:08 UTC (129 KB)
来源:arXiv cs.AI · arxiv.org