通过层级梯度分析改进大语言模型知识编辑：寻找“黄金层”
Golden Layers and Where to Find Them: Improved Knowledge Editing for Large Language Models Via Layer Gradient Analysis
AI 导读
NeurIPS 2026 论文提出层级梯度分析（LGA）方法，通过梯度归因高效定位大语言模型中的“黄金层”，用于知识编辑。研究假设存在固定的黄金层，其编辑效果接近逐样本最优层，且可通过代理数据集识别并泛化到未见查询，避免跨多次编辑运行的反复试错。多个基准数据集实验验证了 LGA 在不同 LLM 类型和知识编辑方法上的有效性与稳健性。
正文 · 原文
Abstract:Knowledge editing in Large Language Models (LLMs) aims to update the model's prediction for a specific query to a desired target while preserving its behavior on all other inputs. This process typically involves two stages: identifying the layer to edit and performing the parameter update. Intuitively, different queries may localize knowledge at different depths of the model, resulting in different sample-wise editing performance for a fixed editing layer. In this work, we hypothesize the existence of fixed golden layers that can achieve near-optimal editing performance similar to sample-wise optimal layers. To validate this hypothesis, we provide empirical evidence by comparing golden layers against ground-truth sample-wise optimal layers. Furthermore, we show that golden layers can be reliably identified using a proxy dataset and generalize effectively to unseen test set queries across datasets. Finally, we propose a novel method, namely Layer Gradient Analysis (LGA) that estimates golden layers efficiently via gradient-attribution, avoiding extensive trial-and-error across multiple editing runs. Extensive experiments on several benchmark datasets demonstrate the effectiveness and robustness of our LGA approach across different LLM types and various knowledge editing methods.
| Comments: | Accepted to NeurIPS 2026 Main Conference |
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2602.20207 [cs.LG] |
| (or arXiv:2602.20207v4 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2602.20207 arXiv-issued DOI via DataCite |
Submission history
From: Anshuman Chhabra [view email]
[v1]
Sun, 22 Feb 2026 22:55:11 UTC (4,290 KB)
[v2]
Fri, 27 Mar 2026 00:35:29 UTC (4,269 KB)
[v3]
Thu, 14 May 2026 19:27:02 UTC (6,768 KB)
[v4]
Wed, 4 Nov 2026 15:20:35 UTC (1,814 KB)
来源：arXiv cs.AI · arxiv.org