FRA：预算受限下的 LoRA 适配器群函数感知压缩方法

View PDF HTML (experimental)

Abstract:Low-Rank Adaptation (LoRA) has become a key mechanism for efficiently specializing foundation models into many task- and user-specific variants. Serving many LoRA adapters under a shared resource budget turns post-hoc compression into a two-level allocation problem: rank must be distributed across layers within each adapter and across adapters within the fleet. Although trained LoRAs contain substantial spectral redundancy, spectral reconstruction alone does not determine how much compression an adapter can functionally tolerate. We formulate fleet compression through a functional rank-distortion frontier for each adapter, followed by fleet-level minimax allocation under a shared rank budget. Given exact frontiers, this decomposition is equivalent to joint optimization over all layer ranks. We introduce Functional Rank Allocation (FRA), an unlabeled method for constructing practical frontiers. FRA measures isolated module truncations end to end, uses these measurements as an additive search surrogate solved by dynamic programming, remeasures every complete candidate, and takes the lower-risk envelope with a complementary spectral proposal before exact minimax allocation over the constructed frontiers. Across 78 adapters from three public fleets, FRA improves worst-case and tail utility over uniform spectral compression in constrained-budget regimes. Fixed-budget probes show that isolated costs rank candidates well, but their calibration shifts across proposal shapes; end-to-end verification and the spectral proposal therefore limit functional-search error.

Subjects:	Artificial Intelligence (cs.AI)
Cite as:	arXiv:2610.36386 [cs.AI]
	(or arXiv:2610.36386v1 [cs.AI] for this version)
	https://doi.org/10.48550/arXiv.2610.36386 arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Shenghui Li [view email]
[v1] Mon, 26 Oct 2026 23:02:12 UTC (50 KB)