arXiv cs.AI· Yuheng Chen, Yufan Chen, Zhuojun Cai, Ruiping Liu, Junwei Zheng, Jiale Wei, Di Wen, Weijia Fan, Kunyu Peng, Jiaming Zhang, Rainer Stiefelhagen·· 3 小时前AI 评分34
Read-Parse-Describe：面向文档解析与视觉元素描述生成的统一数据集
Read, Parse, Describe: Unified Document Parsing with Visual Element Description Generation
AI 导读
一篇 NeurIPS 2026 论文提出 Read-Parse-Describe，一个统一的文档解析与视觉元素描述生成数据集，要求模型对页面图像输出含边界框、语义标签、阅读顺序、文本内容与视觉元素替代文本的有序布局列表。
正文 · 原文
Authors:Yuheng Chen, Yufan Chen, Zhuojun Cai, Ruiping Liu, Junwei Zheng, Jiale Wei, Di Wen, Weijia Fan, Kunyu Peng, Jiaming Zhang, Rainer Stiefelhagen
Abstract:Document understanding systems are typically evaluated as separate components via layout analysis, optical text recognition, reading-order prediction, and image captioning. This separation leaves scientific figures, captions, and visual semantics weakly connected, and fails to measure whether a model can produce a single page-level parse that is both structurally faithful and semantically accessible. We introduce Read-Parse-Describe, a unified dataset for document parsing with visual element description generation. Given a page image, models are required to output an ordered list of layout elements containing bounding boxes, semantic labels, reading-order indices, recognized textual content, and alternative text descriptions for visual elements. The benchmark is built from 2,402 documents, comprising 18,163 continuous pages, 125,218 layout elements, and 11,338 visual instances. Besides, we further propose a plug-and-play space-awareness enhancer that reuses multi-layer visual features and injects grid-aware spatial guidance into vision-language tokens for layout-aware generation. Experiments across end-to-end VLMs and pipeline parsers show that existing systems still struggle to jointly recover reading order and visual semantics, while our enhanced model achieves strong joint performance. These results highlight the need to move beyond isolated document parsing modules toward unified models that jointly read, parse, and describe complex documents. The new dataset and code will be made publicly available at this https URL
| Comments: | Accepted by NeurIPS 2026 |
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.20335 [cs.CV] |
| (or arXiv:2610.20335v2 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2610.20335 arXiv-issued DOI via DataCite |
Submission history
From: Yuheng Chen [view email]
[v1]
Wed, 14 Oct 2026 15:17:28 UTC (3,478 KB)
[v2]
Thu, 12 Nov 2026 14:44:58 UTC (3,474 KB)
来源：arXiv cs.AI · arxiv.org