arXiv cs.AI· Yixuan Xie, Xueqiang Xu, Xinyi Fan, Xinxi Lyu, Tao Feng, Xing Fan, Chenlei Guo, Jie Hao, Shaowen Wang, Jiashuo Sun, Jiawei Han·· 4 小时前AI 评分45
MemComposer：用强化学习为 LLM 智能体组装记忆
MemComposer: Learning to Assemble Memories for LLM Agents via Reinforcement Learning
AI 导读
研究者提出 MemComposer，一种基于强化学习的记忆组装器，可根据当前查询决定为 LLM 智能体选取哪些经验、选多少以及如何排序，并通过基于 GRPO 的 Experience Policy Optimization（ExPO）框架用任务成功反馈优化策略。
正文 · 原文
Authors:Yixuan Xie, Xueqiang Xu, Xinyi Fan, Xinxi Lyu, Tao Feng, Xing Fan, Chenlei Guo, Jie Hao, Shaowen Wang, Jiashuo Sun, Jiawei Han
Abstract:Agent memory enables large language model (LLM) agents to reuse past experiences without updating their parameters. However, semantic similarity alone does not guarantee memory usefulness: downstream performance depends on how memories are selected and presented to the agent. We propose MemComposer, a memory composer that jointly examines the current query and retrieved candidates to decide which experiences to include, how many to select, and how to order them. We train the composer using Experience Policy Optimization (ExPO), a GRPO-based framework that evaluates assembled memory contexts through complete agent interactions and uses task-success feedback to optimize the assembly policy. Across three $\tau^2$-bench domains with two frozen executors, MemComposer outperforms the strongest baseline for each executor while reducing average memory-token usage by 16.9% and 4.9%, respectively. The learned policy also transfers to unseen domains, executors, and memory banks, and benefits from memories generated by stronger models without further training. These results demonstrate the value of learning query-specific memory assembly using downstream task feedback.
| Comments: | 30 pages, 8 figures, 14 tables |
| Subjects: | Artificial Intelligence (cs.AI); Computation and Language (cs.CL) |
| ACM classes: | I.2.7; I.2.6 |
| Cite as: | arXiv:2610.21091 [cs.AI] |
| (or arXiv:2610.21091v2 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.21091 arXiv-issued DOI via DataCite |
Submission history
From: Xueqiang Xu [view email]
[v1]
Thu, 15 Oct 2026 01:47:06 UTC (468 KB)
[v2]
Sat, 24 Oct 2026 01:28:57 UTC (499 KB)
来源：arXiv cs.AI · arxiv.org