arXiv cs.AI· Simiao Ren (Neo), Yuchen Zhou (Neo), Xingyu Shen (Neo), Kidus Zewde (Neo), Tommy Duong (Neo), George Huang (Neo), Hatsanai (Neo), Tiangratanakul (Dennis), Tsang (Dennis), Ng (Dennis), En Wei, Jiayu Xue·· 4 小时前AI 评分53
开源 AI 生成图像检测模型开箱即用表现如何：一项系统性基准研究
How well are open sourced AI-generated image detection models out-of-the-box: A comprehensive benchmark study
AI 导读
一项基准研究对 16 个最先进检测方法的 23 个预训练检测器变体做了零样本评测，覆盖 12 个数据集、260 万张图像和 291 种生成器。
正文 · 原文
Abstract:As AI-generated images proliferate across digital platforms, reliable detection methods have become critical for combating misinformation and maintaining content authenticity. While numerous deepfake detection methods have been proposed, existing benchmarks predominantly evaluate fine-tuned models, leaving a critical gap in understanding out-of-the-box performance -- the most common deployment scenario for practitioners. We present the first comprehensive zero-shot evaluation of 16 state-of-the-art detection methods, comprising 23 pretrained detector variants (due to multiple released versions of certain detectors), across 12 diverse datasets, comprising 2.6~million image samples spanning 291 unique generators including modern diffusion models. Our systematic analysis reveals striking findings: (1)~no universal winner exists, with detector rankings exhibiting substantial instability (Spearman~$\rho$: 0.01 -- 0.87 across dataset pairs); (2)~a 37~percentage-point performance gap separates the best detector (75.0\% mean accuracy) from the worst (37.5\%); (3)~training data alignment critically impacts generalization, causing up to 20--60\% performance variance within architecturally identical detector families; (4)~modern commercial generators (Flux~Dev, Firefly~v4, Midjourney~v7) defeat most detectors, achieving only 18--30\% average accuracy; and (5)~we identify three systematic failure patterns affecting cross-dataset generalization. Statistical analysis confirms significant performance differences between detectors (Friedman test: $\chi^2$=121.01, $p<10^{-16}$, Kendall~$W$=0.524). Our findings challenge the ``one-size-fits-all'' detector paradigm and provide actionable deployment guidelines, demonstrating that practitioners must carefully select detectors based on their specific threat landscape rather than relying on published benchmark performance.
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2602.07814 [cs.CV] |
| (or arXiv:2602.07814v2 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2602.07814 arXiv-issued DOI via DataCite |
Submission history
From: Simiao Ren [view email]
[v1]
Sun, 8 Feb 2026 04:36:13 UTC (200 KB)
[v2]
Sat, 24 Oct 2026 21:40:00 UTC (201 KB)
来源：arXiv cs.AI · arxiv.org