Artifact Distillation：无需训练从前沿模型向小模型迁移能力
Artifact Distillation: Training-Free Transfer from Frontier to Small Language Models
AI 导读
论文提出 Artifact Distillation，让教师模型把解题过程外化成工件库供学生模型在推理时查询，无需梯度更新。
正文 · 原文
Abstract:Closing the gap between frontier and smaller language models is usually approached by distillation: fine-tuning the small model (student) on a frontier model's (teacher) outputs, which binds transfer to training compute and leaves the transferred knowledge largely opaque. We propose Artifact Distillation: a teacher solves training instances and externalizes its problem-solving procedure into an artifact bank that a student queries at inference time, with no gradient updates. Across four LiveBench tasks, a bank written by Claude Opus 4.5 lets Claude Haiku 4.5 outperform unaided Claude Sonnet 4.5 on spatial reasoning (0.707 vs. 0.647) at 60% of its cost, and reach Sonnet 4.5's level on table joining at 89% of its cost. The same raw bank raises Qwen3.5-27B's pooled score from 0.730 to 0.800, showing that transfer is not confined to the teacher's model family.
| Comments: | 10 pages (4 pages main text), 2 figures, 3 tables |
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Machine Learning (cs.LG) |
| Cite as: | arXiv:2610.37680 [cs.CL] |
| (or arXiv:2610.37680v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.37680 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Mustafa Safa Ozdayi [view email]
[v1]
Mon, 12 Oct 2026 20:27:04 UTC (68 KB)
来源：arXiv cs.AI · arxiv.org