arXiv cs.AI· Inish Melroy D Souza, Tom Jackson, Vitor Castro, Rebecca Higginson, Andrea Soltoggio·· 4 小时前AI 评分38
LoRAWeaver:LoRA 表示空间中的方向性适配器路由
LoRAWeaver: Directional Adapter Routing in LoRA Representation Space
AI 导读
LoRAWeaver 提出一种免训练的 LoRA 适配器路由方法,将适配器选择视为适配器输出空间中的表示对齐问题,通过余弦相似度将查询表示与任务原型匹配来选择适配器。在 LLaMA-2-7B 和 LLaMA-2-13B 上针对 48 个任务的 FLAN-v2 基准及分布外任务实验显示,该方法在跨任务设置中优于免训练基线,并接近经训练的检索编码器,且无需训练路由器。
正文 · 原文
Abstract:Large language models are expensive to update through full retraining. Parameter-efficient fine-tuning reduces the cost of adaptation, but repeated adaptation creates a growing pool of task-specific LoRA adapters. This pool complicates adapter selection during inference. Existing routing and encoder-based methods often require retraining when new adapters are added, which limits their use in dynamic settings. This paper treats adapter routing as a representation alignment problem in the adapter-output space. The proposed training-free method selects adapters by matching adapter-conditioned query representations against task prototypes using cosine similarity. Experiments with LLaMA-2-7B and LLaMA-2-13B on a 48-task FLAN-v2 benchmark and on out-of-distribution tasks show that the method outperforms training-free baselines in the cross-task setting and approaches a trained retrieval encoder without learning a router. An ablation confirms the benefit of directional alignment in the adapter-output space.
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2611.17340 [cs.AI] |
| (or arXiv:2611.17340v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2611.17340 arXiv-issued DOI via DataCite |
Submission history
From: Inish Melroy D Souza [view email]
[v1]
Tue, 1 Sep 2026 01:33:50 UTC (492 KB)
来源:arXiv cs.AI · arxiv.org