ScrapeGraphAI-100k:面向 schema 约束 LLM 生成的新数据集
ScrapeGraphAI-100k: Dataset for Schema-Constrained LLM Generation
AI 导读
ScrapeGraphAI-100k 数据集发布,包含 93,695 条经 opt-out 遥测收集的 schema 约束抽取事件,由 9M 原始事件去重并按 schema 平衡而来,覆盖 25,453 个唯一 schema 和 15 种语言(英语占 89%)。
正文 · 原文
Abstract:Producing output that conforms to a specified JSON schema underlies tool use, structured extraction, and knowledge base construction in modern large language models. Despite this centrality, public datasets for the task remain small, synthetic, or text-only, and rarely pair real page content with the prompts and schemas used in practice. We introduce ScrapeGraphAI-100k, 93,695 schema-constrained extraction events collected via opt-out ScrapeGraphAI telemetry in Q2--Q3 2025, deduplicated and balanced by schema from 9M raw events. The corpus spans 25,453 unique schemas, across 15 named languages, with English covering 89% of detected content. Each instance pairs Markdown-converted page content with a prompt, schema, LLM response, and per-example jsonschema-rs structural conformance labels. We characterize structural diversity across the corpus and identify sharp failure thresholds as schema complexity grows. As a case study, a 1.7B student fine-tuned on this data closely tracks the output distribution of its GPT-5-nano teacher, though it still trails a 30B-A3B reference (3.3B active parameters) on schema compliance. We offer this distillation result as preliminary evidence that grounding schema-constrained generation in real practitioner workloads at scale enables training and benchmarking that prior synthetic or text-only corpora could not support. ScrapeGraphAI-100k is publicly available on HuggingFace.
| Subjects: | Information Retrieval (cs.IR); Artificial Intelligence (cs.AI); Computation and Language (cs.CL) |
| Cite as: | arXiv:2602.15189 [cs.IR] |
| (or arXiv:2602.15189v3 [cs.IR] for this version) | |
| https://doi.org/10.48550/arXiv.2602.15189 arXiv-issued DOI via DataCite |
Submission history
From: William Brach [view email]
[v1]
Mon, 16 Feb 2026 20:56:59 UTC (3,783 KB)
[v2]
Fri, 8 May 2026 08:59:17 UTC (758 KB)
[v3]
Thu, 12 Nov 2026 19:54:29 UTC (612 KB)
来源:arXiv cs.AI · arxiv.org