跳到正文
arXiv cs.AI· Vedant Navale·· 1 小时前AI 评分38

MoE-Context:只给模型看它需要的内容

MoE-Context: Show the Model Only What It Needs

AI 导读

arXiv 论文提出 MoE-Context 方法,把上下文分块,每步先用快速判断挑出模型所需的块,其余以单行笔记形式保留在共享 KV cache 中,隐藏而非删除。

正文 · 原文

View PDF HTML (experimental)

Abstract:AI agents read their whole history at every step. People do not work like that: we keep what the moment needs and hold the rest as a rough note we can recall. MoE-Context applies this to language models. The context is split into blocks. Before each step a fast yes or no pass decides which blocks the step needs; the rest stay as one-line notes that can be brought back, and nothing is deleted. In the version we build, one model makes this decision and then answers, over a single key-value cache where hidden blocks are masked rather than removed. On a laptop with Qwen3-1.7B and a synthetic lookup task, showing only the needed block raises accuracy from 93 to 119 of 120, and a masked block is as hidden as a deleted one. The model's own picking is the weak part. A supervised linear probe written into spare output rows lifts its accuracy from 80 to 110 of 120, but simple keyword search picks perfectly on this task, and the probe does not transfer to a second task. We report these results openly, along with two cheap usage patterns that block switching enables: undoing a failed attempt and running subagents without a new context.
Comments: 10 pages, 10 figures
Subjects: Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Machine Learning (cs.LG)
Cite as: arXiv:2611.24379 [cs.AI]
  (or arXiv:2611.24379v1 [cs.AI] for this version)
  https://doi.org/10.48550/arXiv.2611.24379

arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Vedant Navale [view email]
[v1] Wed, 18 Nov 2026 13:08:14 UTC (68 KB)

来源:arXiv cs.AI · arxiv.org