跳到正文
arXiv cs.AI· William Brach, Francesco Zuppichini, Marco Vinciguerra, Lorenzo Padoan·· 4 小时前AI 评分48

ScrapeGraphAI-100k:面向 schema 约束 LLM 生成的新数据集

ScrapeGraphAI-100k: Dataset for Schema-Constrained LLM Generation

AI 导读

ScrapeGraphAI-100k 数据集发布,包含 93,695 条经 opt-out 遥测收集的 schema 约束抽取事件,由 9M 原始事件去重并按 schema 平衡而来,覆盖 25,453 个唯一 schema 和 15 种语言(英语占 89%)。

正文 · 原文

View PDF HTML (experimental)

Abstract:Producing output that conforms to a specified JSON schema underlies tool use, structured extraction, and knowledge base construction in modern large language models. Despite this centrality, public datasets for the task remain small, synthetic, or text-only, and rarely pair real page content with the prompts and schemas used in practice. We introduce ScrapeGraphAI-100k, 93,695 schema-constrained extraction events collected via opt-out ScrapeGraphAI telemetry in Q2--Q3 2025, deduplicated and balanced by schema from 9M raw events. The corpus spans 25,453 unique schemas, across 15 named languages, with English covering 89% of detected content. Each instance pairs Markdown-converted page content with a prompt, schema, LLM response, and per-example jsonschema-rs structural conformance labels. We characterize structural diversity across the corpus and identify sharp failure thresholds as schema complexity grows. As a case study, a 1.7B student fine-tuned on this data closely tracks the output distribution of its GPT-5-nano teacher, though it still trails a 30B-A3B reference (3.3B active parameters) on schema compliance. We offer this distillation result as preliminary evidence that grounding schema-constrained generation in real practitioner workloads at scale enables training and benchmarking that prior synthetic or text-only corpora could not support. ScrapeGraphAI-100k is publicly available on HuggingFace.
Subjects: Information Retrieval (cs.IR); Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
Cite as: arXiv:2602.15189 [cs.IR]
  (or arXiv:2602.15189v3 [cs.IR] for this version)
  https://doi.org/10.48550/arXiv.2602.15189

arXiv-issued DOI via DataCite

Submission history

From: William Brach [view email]
[v1] Mon, 16 Feb 2026 20:56:59 UTC (3,783 KB)
[v2] Fri, 8 May 2026 08:59:17 UTC (758 KB)
[v3] Thu, 12 Nov 2026 19:54:29 UTC (612 KB)

来源:arXiv cs.AI · arxiv.org