投影惩罚梯度追踪:面向通信高效拜占庭鲁棒联邦学习的新框架
Projected Penalty-Gradient Tracking for Communication-Efficient Byzantine-Resilient Federated Learning
AI 导读
研究者在拜占庭攻击与数据异构场景下提出投影惩罚梯度追踪框架,使诚实节点上传压缩的惩罚梯度残差,服务器将重建状态投影到已知惩罚梯度球内,从而在压缩上传下保留拜占庭鲁棒性。
正文 · 原文
Abstract:We investigate communication-efficient federated learning over heterogeneous datasets in the presence of Byzantine adversaries capable of uploading arbitrary information. Penalty-based methods bound the influence of each worker through the bounded gradient of a penalty function; however, the bound is lost once the uploads are compressed, since a Byzantine worker can accumulate an unbounded state at the server. To retain the Byzantine resilience under compressed uploads, we propose a projected penalty-gradient tracking framework, where each honest worker uploads a compressed residual of its penalty gradient and the server projects each reconstructed state onto the known penalty-gradient ball. The framework accommodates any contractive compressor and yields two algorithms: fault-resilient proximal gradient with consistent sparsification (FRPG-CS), whose shared mask is protected by freshly drawn reserve rows, and lazily aggregated FRPG (LAG-FRPG), whose uploads are event-triggered. For smooth and strongly convex losses, we establish a finite-time bound on the expected objective gap, where the Byzantine error floor is independent of the compressor and the compression error decays faster than the optimization error. We also bound the expected number of lazy uploads and derive certificates that separate the finite-penalty model bias from the optimization error. Experiments on MNIST and Fashion-MNIST show that the proposed algorithms attain the highest accuracy among twelve methods under the Min-Max attack, whereas a compressed-momentum benchmark and a local-update benchmark remain preferable in attack-free accuracy and total traffic, respectively.
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2611.22929 [cs.LG] |
| (or arXiv:2611.22929v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2611.22929 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Yanjie Dong [view email]
[v1]
Tue, 17 Nov 2026 15:28:03 UTC (116 KB)
来源:arXiv cs.AI · arxiv.org