arXiv 论文提出 Mechanistic Shaping 框架，通过内部表征塑造并监测模型
Shaping & Monitoring Models through their Internal Representations
Thomas Fel 等人在 arXiv 发表论文（arXiv:2611.07071），提出 Mechanistic Shaping 框架，在训练中通过内部表征让模型遵循规范，控制其表达和依赖的概念。
论文提出用模型内部表征在训练期约束行为，并给出监测概念是否被规避的三种几何机制，对可解释性训练有方法参考价值。
Authors:Thomas Fel, Matthew Kowal, Matt Feiszli, Satchel Grant, Andrew Lee, Chris Earls, Fatih Dinc, Vasudev Shyam, Usha Bhalla, Leon Bergen, Siddharth Boppana, Ching Fang, Michael Byun, Tristan Gallent, Nicholas Wang, Tal Haklay, Dron Hazra, Neehar Kondapaneni, Connor Watts, Ryan Panwar, Thomas Dooms, Jack Merullo, Atticus Geiger, Ekdeep Singh Lubana, Thomas McGrath, Owen Lewis
Abstract:We propose Mechanistic Shaping, a unified framework for shaping models through their internal representations. Mechanistic Shaping is applied during training so that models follow specifications (rules) about which concepts they express, measured via their outputs or internal representations (e.g., no harmful content), and also which concepts they rely on, i.e., what they use to make their decisions (e.g., not a candidate's gender when screening a résumé). Across multiple domains (vision, medical imaging, language), scales (up to 32B parameters), and training paradigms (classification, SFT, DPO, GRPO), we find that Mechanistic Shaping substantially reduces targeted behaviors while preserving task performance. Since training against a representation can also modify it, we ask whether shaped concepts remain monitorable, a question that depends on the class of monitors considered. We therefore introduce a hierarchy of such classes, which separates a concept that escapes one probe from a concept that is truly gone. We characterize three geometric mechanisms of circumvention: the concept can drift away from the monitor, be erased from the layer, or persist in higher-order statistics as a nonlinear encoding, which we explain through a product-code construction. Although other mechanisms may arise in different settings or at larger scales, these three account for all of our observations, and each can be detected by a matching monitor. In practice, we find that shaped concepts largely remain monitorable: refitted linear probes recover the concept in most settings, and the original probe remains reliable whenever gradients do not flow through it. Taken together, these results point to the potential of using interpretability as a tool for training, and call for a correspondingly rigorous science of interpretability-based training.
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2611.07071 [cs.LG] |
| (or arXiv:2611.07071v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2611.07071 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Thomas Fel [view email]
[v1]
Wed, 4 Nov 2026 19:29:13 UTC (19,265 KB)
来源：arXiv cs.AI · arxiv.org