跳到正文
arXiv cs.AI· Inish Melroy D Souza, Tom Jackson, Vitor Castro, Rebecca Higginson, Andrea Soltoggio·· 4 小时前AI 评分38

LoRAWeaver:LoRA 表示空间中的方向性适配器路由

LoRAWeaver: Directional Adapter Routing in LoRA Representation Space

AI 导读

LoRAWeaver 提出一种免训练的 LoRA 适配器路由方法,将适配器选择视为适配器输出空间中的表示对齐问题,通过余弦相似度将查询表示与任务原型匹配来选择适配器。在 LLaMA-2-7B 和 LLaMA-2-13B 上针对 48 个任务的 FLAN-v2 基准及分布外任务实验显示,该方法在跨任务设置中优于免训练基线,并接近经训练的检索编码器,且无需训练路由器。

正文 · 原文

View PDF HTML (experimental)

Abstract:Large language models are expensive to update through full retraining. Parameter-efficient fine-tuning reduces the cost of adaptation, but repeated adaptation creates a growing pool of task-specific LoRA adapters. This pool complicates adapter selection during inference. Existing routing and encoder-based methods often require retraining when new adapters are added, which limits their use in dynamic settings. This paper treats adapter routing as a representation alignment problem in the adapter-output space. The proposed training-free method selects adapters by matching adapter-conditioned query representations against task prototypes using cosine similarity. Experiments with LLaMA-2-7B and LLaMA-2-13B on a 48-task FLAN-v2 benchmark and on out-of-distribution tasks show that the method outperforms training-free baselines in the cross-task setting and approaches a trained retrieval encoder without learning a router. An ablation confirms the benefit of directional alignment in the adapter-output space.
Subjects: Artificial Intelligence (cs.AI)
Cite as: arXiv:2611.17340 [cs.AI]
  (or arXiv:2611.17340v1 [cs.AI] for this version)
  https://doi.org/10.48550/arXiv.2611.17340

arXiv-issued DOI via DataCite

Submission history

From: Inish Melroy D Souza [view email]
[v1] Tue, 1 Sep 2026 01:33:50 UTC (492 KB)

来源:arXiv cs.AI · arxiv.org