arXiv cs.AI· Biprarshi Biswas, Arya Anshuman Ojha, Yubaraj Das, Devansh Khosla, Kamran Rais, Saankhya Samanta, Subhamoy Maitra, Pantelimon Stanica·· 3 小时前AI 评分39
提取结合蒸馏：用密码分析技术从深度神经网络中恢复参数
Extraction meets Distillation - cryptanalytic techniques to recover parameters from a Deep Neural Network
AI 导读
一项 arXiv 研究将密码分析的权重方向提取与基于预测标签的学习相结合，重建全连接神经网络，并将 ReLU 提取实现改造适配 Leaky ReLU。在最大合成架构的 8 个搜索种子上，Leaky ReLU 在最深层隐藏层恢复 51–57 个神经元而 ReLU 为零，但该优势并非在所有架构和层都成立。
正文 · 原文
Abstract:We study fully connected neural-network reconstruction by combining cryptanalytic weight-direction extraction with learning from predicted labels. We adapt a ReLU extraction implementation to Leaky ReLU, compare sign-search methods, and estimate biases and output layers. We prove conditional bounds for direction recovery with approximate prefixes, preservation of local distance votes, prediction changes from aligned parameter errors, and sign selection with independent scoring data. We also specify exact row restoration and distinguish its guarantee from that of gradient masking. The experiments evaluate assisted extraction, with privileged access to gradients, hidden activations, and victim parameters, followed by label-based reconstruction. Across eight search seeds on the largest synthetic architecture, Leaky ReLU recovers 51--57 neurons in the deepest hidden layer, while ReLU recovers none. This advantage does not hold uniformly across architectures and layers. Removing victim-weight access from the sign walk gives less accurate signs, despite high final prediction agreement. Across 40 seeded runs on the two smallest synthetic victims, agreement on 5,000 inputs excluded from fitting and selection ranges from $0.9890$ to $0.9992$. On two CIFAR-10 victims, the reconstructed models achieve $56.40\%$ and $58.29\%$ agreement and exceed the distillation baseline on three functional metrics. We report label requests separately from privileged access and explain why prediction agreement does not identify network parameters.
| Comments: | 39 pages, no figure |
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Cryptography and Security (cs.CR) |
| MSC classes: | 68P25 |
| ACM classes: | E.3 |
| Cite as: | arXiv:2610.33671 [cs.LG] |
| (or arXiv:2610.33671v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2610.33671 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Subhamoy Maitra [view email]
[v1]
Sat, 24 Oct 2026 13:36:49 UTC (58 KB)
来源：arXiv cs.AI · arxiv.org