Transformer 范式:动态处理与新的推理原理
The Transformer Paradigm: Dynamic Processing and a New Principle of Inference
AI 导读
arXiv 论文提出,Transformer 革命并非深度学习的单纯量化扩张,而是基于全新推理原理的范式转变。作者将 token 表示理解为被参数化“变换概念”动态改造的系统,推理时部分变换由当前输入动态构建,其机械结构基础是输出权重互连,称为序列级交互式动态并行处理(SIDPP)。该框架可解释模型对提示词的强敏感性,并为可解释性、控制、替代架构、神经形态系统等方向开辟研究路径。
正文 · 原文
Abstract:We argue that the Transformer revolution is not adequately understood as a merely quantitative expansion of deep learning, but as the emergence of a new paradigm based on a different principle of inference. We interpret the Transformer as a system in which token representations are transformed by parameterized transformations, or transforming concepts. During inference, some of these transformations are dynamically constructed from the current input: previously acquired transformations are first adapted and then applied. At the mechanical-structural level, this dynamic processing is realized through output-weight interconnections, whereby outputs of some networks determine the weights of others. The resulting mode of processing, which we call Sequence-level Interactive Dynamic Parallel Processing (SIDPP), provides a framework for explaining strong prompt sensitivity and opens research directions concerning interpretability, control, alternative AI architectures, neuromorphic systems, and possible biological realizations of dynamic processing.
| Comments: | v1: 15 pp., 5 figs., 1 table |
| Subjects: | Artificial Intelligence (cs.AI); Hardware Architecture (cs.AR); Computers and Society (cs.CY); Emerging Technologies (cs.ET); Neural and Evolutionary Computing (cs.NE) |
| Cite as: | arXiv:2611.17953 [cs.AI] |
| (or arXiv:2611.17953v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2611.17953 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Marco Giunti [view email]
[v1]
Fri, 13 Nov 2026 09:41:34 UTC (4,165 KB)
来源:arXiv cs.AI · arxiv.org