联邦零阶优化的高效精确遗忘方法 FedZOUL
Efficient Exact Unlearning for Federated Zeroth-Order Optimization
AI 导读
一项 arXiv 研究针对联邦零阶优化（FedZOO）提出精确遗忘方案，利用算法的总变差稳定性使删除前后模型分布重叠，从而复用计算。研究设计了 FedZOO-S 与 FedZOO-NS 两种稳定学习算法及对应的精确遗忘流程，证明遗忘后模型与重训练具有相同联合分布，累积遗忘时间最多比重新优化减少 56.1%，与一阶基线相比通信开销最多降低约四个数量级。
正文 · 原文
Abstract:Federated zeroth-order optimization (FedZOO) reduces the memory and communication costs of collaborative training for edge intelligence by avoiding backpropagation while communicating only random seeds and scalar coefficients. However, existing methods lack efficient mechanisms for exact unlearning to fully remove the influence of training data when users request deletion. A key insight is that total variation stability of the learning algorithm creates overlap between model distributions before and after data deletion, allowing exact unlearning to reuse computation through coupling. Based on this observation, we jointly design two stable zeroth-order learning algorithms and their corresponding exact-unlearning procedures: \textsc{FedZOO-S} obtains stability through client and mini-batch subsampling, enabling \textsc{FedZOUL-S} to perform efficient unlearning by preserving earlier updates and restarting from the first affected step. \textsc{FedZOO-NS} further combines subsampling with Gaussian noise to increase overlap between update distributions; \textsc{FedZOUL-NS} exploits this overlap through rejection--reflection coupling to reuse affected updates when the coupling is accepted. We prove exact unlearning for both algorithms: the model and saved state have the same joint distribution as retraining with the corresponding learner on retained data. We further establish stability and convergence guarantees for the learning algorithms and derive upper bounds on expected unlearning cost. Experiments demonstrate effective unlearning with competitive model utility and up to $56.1\%$ less cumulative unlearning time than re-optimization. Compared with first-order baselines, our methods reduce communication overhead by up to approximately four orders of magnitude.
| Comments: | 36 pages, 8 figures |
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Distributed, Parallel, and Cluster Computing (cs.DC); Networking and Internet Architecture (cs.NI) |
| Cite as: | arXiv:2610.35211 [cs.LG] |
| (or arXiv:2610.35211v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2610.35211 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Weijun Yin [view email]
[v1]
Mon, 26 Oct 2026 11:03:48 UTC (1,777 KB)
来源：arXiv cs.AI · arxiv.org