上下文学习与上下文外幻觉的机制分析
A Mechanistic Analysis of In-Context Learning and Out-of-Context Hallucination
AI 导读
一项 arXiv 研究用动态上下文中功能角色的统一视角,分析大语言模型的上下文学习(ICL)与上下文外(OOC)幻觉两种看似对立的行为。研究推导出一阶分解刻画角色效应,发现其显著影响取决于贡献量级充足和带符号聚合一致两个必要条件;因果分析表明角色区间贡献影响响应结构、内容选择与事实取值等关键 token 决策。当角色区间支持不足时决策会失败,而补足支持可逆转失败结果。
正文 · 原文
Abstract:Large language models can infer and follow specific rules through in-context learning (ICL), yet may fail to remain faithful to provided evidence, giving rise to out-of-context (OOC) hallucination. Existing work has largely investigated the underlying mechanisms of ICL and OOC hallucination along separate lines. However, this separation obscures the mechanistic basis for these seemingly opposing outcomes. In this work, we study this contrast through the unified view of functional roles in dynamic context. The key observation is that role information can be internally represented without reliably constraining output. We derive a first-order decomposition to characterize the role effect, showing that its substantial influence depends on two necessary conditions, namely sufficient contribution magnitude and coherent signed aggregation. Causal analysis further shows that role span contributions affect key token choices involving response structure, content selection, and factual values. We further show that role-consistent choices depend on sufficient role span support at these key decisions, while failing choices can be reversed when this support overcomes the deficit. These findings support a common mechanistic view of ICL and OOC hallucination: each functional role defines the pattern of output decisions over which role span contributions must remain effective, and differences in this organization account for the seemingly opposing outcomes.
| Comments: | 25 pages, 6 figures |
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Machine Learning (cs.LG) |
| Cite as: | arXiv:2611.24000 [cs.CL] |
| (or arXiv:2611.24000v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2611.24000 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Tianxiang Chen [view email]
[v1]
Wed, 18 Nov 2026 07:48:46 UTC (2,266 KB)
来源:arXiv cs.AI · arxiv.org