arXiv cs.AI· Yaxin Zhang, Xingyu Wu, Yuchen Yan, Yuqin Dai, Hong Zhao, Lixing Shen, Zhan Chen, Jian Shao, Yongliang Shen·· 3 小时前AI 评分37
SimRec：从模拟交互中学习搜索、询问与推荐的强化学习框架
SimRec: Learning to Search, Ask, and Recommend from Simulated Interaction
AI 导读
SimRec 是一个强化学习框架，端到端训练工具集成推荐智能体的完整交互策略，包含任务锚定的用户模拟器和基于 GRPO 的任务对齐奖励。在 ESCI 和 Amazon C4 上，它将 Qwen3-4B 智能体的 NDCG@100 从 32.23% 提升至 35.42%、从 22.94% 提升至 24.72%，超过更大的免训练智能体。
正文 · 原文
Abstract:LLM-based recommendation agents can search catalogs, inspect products, ask users, and revise rejected recommendations. However, these behaviors are often governed by hand-designed workflows, while existing training methods optimize isolated components such as query rewriting or list ranking rather than the interaction policy as a whole. We present SimRec, a reinforcement learning framework that trains the full interaction policy of a tool-integrated recommendation agent. SimRec introduces two components. A task-anchored user simulator provides feedback on every recommendation: conditioned on the current task need rather than on user profile, and it can be an external LLM or the recommender itself under self-play. A task-aligned reward optimizes the whole trajectory with GRPO, crediting natural retrieval quality and valid tool use alongside the final hit, with training-only target injection to handle sparse hits over large catalogs. On ESCI and Amazon C4, SimRec improves the NDCG@100 of a Qwen3-4B agent from 32.23% to 35.42% and from 22.94% to 24.72%}, outperforming training-free agents on far larger backbones and task-aligned baselines on NDCG. Analysis shows that the trained agent verifies candidates, uses preferences, and recovers from rejected recommendations far more often than its untrained counterpart. Our code is available at this https URL .
| Comments: | 19 pages,5 figures,16 tables. Code: this https URL |
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Information Retrieval (cs.IR) |
| Cite as: | arXiv:2611.15426 [cs.CL] |
| (or arXiv:2611.15426v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2611.15426 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Yaxin Zhang [view email]
[v1]
Wed, 11 Nov 2026 12:53:52 UTC (1,210 KB)
来源：arXiv cs.AI · arxiv.org