跳到正文
arXiv cs.AI· Shiguang Wu, Yaqing Wang, Quanming Yao·· 2 小时前AI 评分40

智能体上下文学习:Transformer 在闭环交互中能学到哪些算法

Agentic In-Context Learning: What Algorithms Can Transformers Learn in Closed-Loop Interaction?

AI 导读

一篇 arXiv 论文提出 Agentic In-Context Learning(智能体上下文学习)框架,研究固定参数 Transformer 在交互闭环中的学习算法视角,用状态、更新与决策规则刻画主动学习、bandits 和在线 RL。

正文 · 原文

View PDF HTML (experimental)

Abstract:The learning algorithm view of in-context learning (ICL) studies how fixed-parameter Transformers learn from supplied examples. Language model agents extend this setting through interaction: their actions determine what evidence enters context next. We call this regime Agentic In-Context Learning (Agentic ICL). Existing studies largely depend on externally supplied contexts or reward-bearing interaction histories, while work on general tool-using agents lacks a learning algorithm view. We represent an interactive learner by its state, update, and decision rule, covering active learning, bandits, and online reinforcement learning (RL). For expressivity, we construct causal Transformers that approximate these algorithms over a finite horizon, and bound the resulting loss in return. For training, we characterize when optimizing process rewards preserves behavior that maximizes the final outcome. Distillation experiments show that Transformers can learn to execute three representative recursions. We then train models from outcome or process rewards without oracle action or state targets, as in RL post-training. Different rewards yield distinct decision policies. Improving the alignment of process supervision with outcome-optimal action preferences shifts learned decisions accordingly. Such behavioral differences persist in a pretrained language model across textual paraphrases.
Subjects: Artificial Intelligence (cs.AI)
Cite as: arXiv:2610.23051 [cs.AI]
  (or arXiv:2610.23051v2 [cs.AI] for this version)
  https://doi.org/10.48550/arXiv.2610.23051

arXiv-issued DOI via DataCite

Submission history

From: Shiguang Wu [view email]
[v1] Fri, 16 Oct 2026 07:28:42 UTC (321 KB)
[v2] Tue, 17 Nov 2026 06:38:29 UTC (322 KB)

来源:arXiv cs.AI · arxiv.org