Recursive SALM：为预训练语言模型引入持久潜在状态的递归自适应框架
Recursive State-Augmented Language Models: Persistent Latent-State Adaptation of Pretrained Language Models
AI 导读
arXiv 新论文提出 Recursive State-Augmented Language Models（Recursive SALM）框架，通过随每次观察演化的持久潜在状态来生成对预训练语言模型的自适应。
正文 · 原文
Abstract:Language models rely on finite token contexts, limiting access to earlier interactions. We introduce Recursive State-Augmented Language Models (Recursive SALM), a framework in which a persistent latent state evolves with each observation and generates adaptations of a pretrained language model. A modular read-transition-write architecture separates observation encoding, latent dynamics, and predictor adaptation. We develop deterministic and stochastic formulations.
The experiments evaluate only deterministic SALM with frozen GPT-2 small and exclude earlier evidence from its decoder context. On a four-turn hypothesis-inference task, latent-gated LoRA reaches 89.1% accuracy, comparable to 88.0% for a trained full-context LoRA baseline, with earlier evidence available solely through a 64-dimensional state. Zeroing or shuffling the state substantially reduces accuracy. In a two-turn task with codewords unseen in training, several interfaces improve candidate ranking over chance, but rarely generate the exact codeword under the evaluated training schedule. These experiments show that persistent latent-conditioned adaptation can support sequential inference when earlier evidence is absent from the decoder context.
| Comments: | 13 pages, 2 figures. Preprint |
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Machine Learning (stat.ML) |
| Cite as: | arXiv:2611.16375 [cs.LG] |
| (or arXiv:2611.16375v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2611.16375 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Omer Haq [view email]
[v1]
Thu, 12 Nov 2026 06:42:54 UTC (128 KB)
来源：arXiv cs.AI · arxiv.org