GPT4o-Receipt：AI 生成单据取证数据集与人类对比研究

View PDF HTML (experimental)

Abstract:Can humans detect AI-generated financial documents better than machines? We present GPT4o-Receipt, a benchmark of 1,235 receipt images pairing GPT-4o-generated receipts with authentic ones from established datasets, evaluated by five state-of-the-art multimodal LLMs and a 30-annotator crowdsourced perceptual study. Our findings reveal a striking paradox: humans are better at seeing AI artifacts, yet worse at detecting AI documents. Human annotators exhibit the largest visual discrimination gap of any evaluator, yet their binary detection F1 falls well below Claude Sonnet 4 and below Gemini 2.5 Flash. This paradox resolves once the mechanism is understood: the dominant forensic signals in AI-generated receipts are arithmetic errors -- invisible to visual inspection but systematically verifiable by LLMs. Humans cannot perceive that a subtotal is incorrect; LLMs verify it in milliseconds. Beyond the human--LLM comparison, our five-model evaluation reveals dramatic performance disparities and calibration differences that render simple accuracy metrics insufficient for detector selection. GPT4o-Receipt, the evaluation framework, and all results are released publicly to support future research in AI document forensics.

Comments:	12 pages, 7 figures, 7 tables
Subjects:	Artificial Intelligence (cs.AI); Computer Vision and Pattern Recognition (cs.CV)
ACM classes:	I.4.9; I.2.10
Cite as:	arXiv:2603.11442 [cs.AI]
	(or arXiv:2603.11442v3 [cs.AI] for this version)
	https://doi.org/10.48550/arXiv.2603.11442 arXiv-issued DOI via DataCite

Submission history

From: Simiao Ren [view email]
[v1] Thu, 12 Mar 2026 02:05:27 UTC (94 KB)
[v2] Tue, 24 Mar 2026 22:18:12 UTC (94 KB)
[v3] Sat, 24 Oct 2026 22:21:01 UTC (97 KB)