零样本权重空间编程:用 LLM 手写前馈网络全部参数预测结构化数据集目标变量
Zero-Shot Weight-Space Programming for Prediction and Classification of Target Variables in Structured Datasets
AI 导读
一项 arXiv 研究让大语言模型零样本完成权重空间编程:仅查看小数据集与前馈网络架构,直接写出全部权重和偏置的具体数值,不用优化器、梯度或代码执行,并预先注册预期得分。冻结网络在最优解为线性处达到闭式最优,在三个非线性层级的生成过程不可约误差处持平,其中两个超过包括梯度下降训练的同架构在内的所有拟合基线;两个失败层级均因目标结构相对分析坐标呈斜向。
正文 · 原文
Abstract:Weight-space programming fixes every parameter of a feed-forward network by hand, with no gradient descent anywhere in the construction. Every demonstration so far starts from a target function handed to the model as a rule system, making it a compilation problem. A supervised dataset withholds that specification, so the procedure must do the statistical inference itself and then express the result in the parameterisation it was given. We ask whether a large language model can do both. It is shown a small dataset in full, with the architecture of a feed-forward network, and writes out every weight and bias as a literal number, with no optimizer, no gradient, no code execution, and no chance to evaluate it first. Each construction is sealed and pre-registers its expected score. The graded suite runs from two toy problems, through nine synthetic problems disguised by a hidden rotation, to the classical benchmarks and one real engineering regression. The frozen network matches the closed-form optimum wherever that optimum is linear, reaches the irreducible error of the generating process on three rungs where it is not, and beats every fitted baseline on two of those, including the same architecture trained by gradient descent. It fails on two rungs, both times because the structure to be found is oblique to the coordinates it analysed. Three further constructors separate two failure modes a single score conflates: one returns no parameter vector at all on the hardest rungs, another delivers everywhere while writing networks worse than predicting the mean. One of those failures proves avoidable: another constructor solves the oblique-hinge rung by a margin that survives multiple-comparison correction. What predicts whether the hidden layer earns its keep is the training error of a linear separator, which the composite complexity index misses.
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2611.19292 [cs.AI] |
| (or arXiv:2611.19292v2 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2611.19292 arXiv-issued DOI via DataCite |
Submission history
From: Eduardo C. Garrido-Merchán [view email]
[v1]
Sat, 14 Nov 2026 09:37:17 UTC (79 KB)
[v2]
Tue, 17 Nov 2026 09:50:04 UTC (80 KB)
来源:arXiv cs.AI · arxiv.org