跳到正文
arXiv cs.AI· Truong Xuan Khanh·· 1 小时前AI 评分39

响应相关信息不足以判定神经网络训练状态等价性

Response-Relevant Information Is Not Sufficient for Neural Training-State Equivalence

AI 导读

一项 arXiv 研究提出操作性定义:两个检查点若在训练扰动后继续训练时响应相似,才算训练状态等价。

正文 · 原文

View PDF HTML (experimental)

Abstract:Response-relevant information is not sufficient evidence that two neural networks are in the same training state. We define training-state equivalence operationally: two checkpoints are equivalent, relative to an intervention family and horizon, if they respond alike when training is perturbed and continues. On fresh runs of a one-layer transformer learning modular addition, response to a weight-decay pulse is highly reproducible relative to between-checkpoint differences (rho_state = 0.934 at 500 steps), establishing the target as measurable. Fifteen descriptors of weight spectra, representation geometry and optimiser geometry carry within-run response-relevant information beyond weight norm and loss on a development dataset, predicting held-out response deviation across seed blocks. But their apparent 16.1% advantage at matching checkpoints by future response does not separate from a conditional placebo preserving each descriptor's fitted dependence on the conventional observables, and a calibrated positive control makes simple insensitivity an unlikely explanation. Information a decoder can extract need not be organised in the pairwise geometry that similarity judgments require.
Comments: 18 pages, 5 figures. Code and provenance verified artifacts: this https URL
Subjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI)
Cite as: arXiv:2611.23245 [cs.LG]
  (or arXiv:2611.23245v1 [cs.LG] for this version)
  https://doi.org/10.48550/arXiv.2611.23245

arXiv-issued DOI via DataCite

Submission history

From: Xuan Khanh Truong [view email]
[v1] Fri, 4 Sep 2026 05:49:16 UTC (111 KB)

来源:arXiv cs.AI · arxiv.org