两层 ReLU 网络回归良性过拟合研究论文已被作者撤回
Benign Overfitting for Regression with Trained Two-Layer ReLU Networks
AI 导读
Junhyung Park 在 arXiv 上撤回了编号 2410.06191 的论文,该论文原研究经梯度流训练的两层全连接 ReLU 网络在最小二乘回归中的泛化与良性过拟合。作者撤稿时声明证明存在致命错误,v2 版本已于近日撤回,无可用的更新版本。
正文 · 原文
This paper has been withdrawn by Junhyung Park
No PDF available, click to view other formats
Abstract:We study the least-square regression problem with a two-layer fully-connected neural network, with ReLU activation function, trained by gradient flow. Our first result is a generalization result, that requires no assumptions on the underlying regression function or the noise other than that they are bounded. We operate in the neural tangent kernel regime, and our generalization result is developed via a decomposition of the excess risk into estimation and approximation errors, viewing gradient flow as an implicit regularizer. This decomposition in the context of neural networks is a novel perspective of gradient descent, and helps us avoid uniform convergence traps. In this work, we also establish that under the same setting, the trained network overfits to the data. Together, these results, establishes the first result on benign overfitting for finite-width ReLU networks for arbitrary regression functions.
| Comments: | There were fatal errors in the proofs. For the updated version, see https://arxiv.org/abs/2505.11621 |
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Machine Learning (stat.ML) |
| Cite as: | arXiv:2410.06191 [cs.LG] |
| (or arXiv:2410.06191v2 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2410.06191 arXiv-issued DOI via DataCite |
Submission history
From: Junhyung Park [view email]
[v1]
Tue, 8 Oct 2024 16:54:23 UTC (73 KB)
[v2]
Fri, 13 Nov 2026 16:07:18 UTC (1 KB) (withdrawn)
来源:arXiv cs.AI · arxiv.org