arXiv 论文提出自动批量挖掘带意外副作用的训练数据集
Mass Producing Training Data with Surprising Side Effects
AI 导读
arXiv 论文提出 DSSE 概念与自动化流水线，用于批量发现会让语言模型产生训练域外意外行为的数据集，如不安全代码数据诱发广泛失对齐。研究用 LLM 评审定义三个目标，经进化、强化学习和 subliminal learning 方法搜索，产出覆盖 8 种行为的 6,575 个数据集，许多可跨模型家族和规模泛化；作者称前沿模型在预测副作用上优于人类。
正文 · 原文
Abstract:Training data can have unexpected side effects on language model behavior outside the training domain: insecure code data can induce broad misalignment, and outdated bird names can induce a 19th-century persona. Understanding such generalization is hard because only dozens of isolated examples exist, each constructed by hand or discovered by accident. We propose automatically discovering such datasets with surprising side effects (DSSEs). We define three objectives for DSSEs: they should (1) elicit a target behavior, (2) read like ordinary post-training data, and (3) be surprising to human auditors reading the data. We operationalize each with LLM judges, and search for high-scoring datasets via evolution, reinforcement learning, and subliminal learning approaches. After re-scoring and filtering with held-out metrics, our pipeline produces a collection of 6,575 DSSEs across 8 behaviors, many of which generalize across model families and sizes. We find our automated metrics track ML researchers' judgments, and frontier models outperform humans at predicting side effects. These datasets form a testbed for studying generalization and lay the groundwork for predicting training side effects before model deployment. Explore them at this https URL.
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Machine Learning (cs.LG) |
| Cite as: | arXiv:2611.06324 [cs.CL] |
| (or arXiv:2611.06324v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2611.06324 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Yaowen Ye [view email]
[v1]
Wed, 4 Nov 2026 10:57:28 UTC (1,040 KB)
来源：arXiv cs.AI · arxiv.org