FS-EWC：面向无回放持续学习的 Fisher 形塑弹性权重巩固方法
Beyond Parameter Protection: Fisher-Shaped EWC for Replay-Free Continual Learning
AI 导读
arXiv 论文提出 FS-EWC（Fisher-Shaped EWC），将 Fisher 信息矩阵（FIM）的作用从参数保护扩展到解的形塑。该方法在任务学习中最小化邻域内最坏情况的 FIM 迹，并在任务结束时用基于轨迹的模型合并构建保留解。多基准实验显示 FS-EWC 优于 EWC，稳定性—可塑性平衡更佳，对参数漂移容忍度更高。
正文 · 原文
Abstract:In replay-free continual learning, Elastic Weight Consolidation (EWC) uses Fisher information matrix (FIM) to protect acquired knowledge by penalizing changes to historically important parameters. However, a local importance estimate at the retained solution does not in general characterize sensitivity after finite parameter displacement. This motivates controlling Fisher sensitivity over a neighborhood of potential future updates. We propose Fisher-Shaped EWC (FS-EWC), which extends the role of FIM from parameter protection to solution formation for future retention. During task learning, FS-EWC minimizes the worst-case empirical FIM trace within a prescribed neighborhood, explicitly favoring solutions that are less sensitive to bounded future displacement. At task completion, trajectory-based model merging constructs the retained solution from multiple optimization states rather than a single terminal iterate, after which FIM is estimated for subsequent consolidation. Experiments across multiple benchmarks show that FS-EWC improves performance over EWC, while controlled studies indicate a more favorable stability--plasticity balance and greater tolerance to parameter drift. These results suggest that effective knowledge retention requires both forming solutions with future retention in mind and protecting acquired knowledge.
| Comments: | 9 pages, 7 figures, conference |
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.37145 [cs.LG] |
| (or arXiv:2610.37145v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2610.37145 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Liying Wang [view email]
[v1]
Tue, 27 Oct 2026 12:42:27 UTC (1,168 KB)
来源：arXiv cs.AI · arxiv.org