LAPS：为大语言模型持续适配学习锚定提示词空间
Learning an Anchored Prompt Space for Continual Adaptation of Large Language Models
AI 导读
一篇 arXiv 论文提出 LAPS（Learning an Anchored Prompt Space），通过自蒸馏将历史提示词与新模型对齐，并用可学习的 Bézier 控制提示词构建锚定提示词空间，实现任务间正向迁移。在 TRACE 基准上跨三个 Qwen3 模型规模实验显示，LAPS 一致优于基于蒸馏、提示词及提示词-参数联合适配方法，提升整体任务表现并减少遗忘。
正文 · 原文
Abstract:Continually adapting large language models requires acquiring new knowledge while preserving previously learned capabilities. Although jointly adapting model parameters and task-specific soft prompts is promising, it has two key limitations: previously learned prompts may lose effectiveness as the model evolves, and relationships among prompts that could support cross-task transfer are not explicitly learned. We propose Learning an Anchored Prompt Space (LAPS), which preserves historical prompt effectiveness and learns relationships among task-specific soft prompts to facilitate positive transfer. Historical prompts are first aligned with the updated model through self-distillation. LAPS then constructs an anchored prompt space whose vertices correspond to learned task-specific soft prompts and whose intermediate geometry is shaped by learnable Bézier control prompts. Once this space is learned, LAPS identifies the best-performing prompt for each task on its validation set, allowing the selected prompt to draw on knowledge acquired from related tasks. Experiments on the TRACE benchmark across three Qwen3 model scales show that LAPS consistently outperforms existing distillation-based, prompt-based, and joint prompt-parameter adaptation methods, improving overall task performance while reducing forgetting.
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2609.32499 [cs.CL] |
| (or arXiv:2609.32499v2 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2609.32499 arXiv-issued DOI via DataCite |
Submission history
From: Rongguang Ye [view email]
[v1]
Sat, 26 Sep 2026 11:39:55 UTC (3,278 KB)
[v2]
Tue, 27 Oct 2026 13:08:41 UTC (3,330 KB)
来源：arXiv cs.AI · arxiv.org