Merge and Branch：高效面向推荐的 LLM 树状结构监督微调方法
Merge and Branch: Efficient Tree-Structured Supervised Fine-Tuning of LLMs for Recommendation
AI 导读
arXiv 论文提出 Merge and Branch（M&B）方法，降低基于 LLM 的生成式推荐在监督微调中的输入开销。Merge 将历史物品的语义标识符和文本压缩为少量学习 token，Branch 基于压缩历史预测多个目标 SID，并以浅宽树结构组织样本，配合 tree FlashAttention 和树状 KV cache 实现端到端训练与约束解码。
正文 · 原文
Abstract:LLM-based generative recommendation reads long, text-rich interaction histories but generates only short semantic identifiers (SIDs). This asymmetry makes supervised fine-tuning inefficient: encoding the history is expensive, yet each history usually provides supervision for only one target. We present Merge and Branch (M&B) to reduce input cost and learn more from each encoded history. Merge compresses each historical item's SID and text into a small number of learned tokens. Branch uses the compressed history to predict multiple target SIDs. To share history computation while keeping these targets independent, M&B organizes each example as a shallow, wide tree. We develop tree FlashAttention to execute this layout efficiently, processing the shared history once and propagating gradients from every branch for end-to-end training. A tree-structured key/value cache extends history sharing to constrained decoding. Experiments on public benchmarks show that M&B outperforms strong recommendation baselines while substantially improving training and decoding efficiency.
| Comments: | 12 pages, 8 figures |
| Subjects: | Information Retrieval (cs.IR); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.37973 [cs.IR] |
| (or arXiv:2610.37973v1 [cs.IR] for this version) | |
| https://doi.org/10.48550/arXiv.2610.37973 arXiv-issued DOI via DataCite |
Submission history
From: Yunqin Zhu [view email]
[v1]
Tue, 27 Oct 2026 21:10:11 UTC (741 KB)
来源：arXiv cs.AI · arxiv.org