TokenMinds：用预训练用户 token 与嵌入实现大规模推荐系统的用户理解
TokenMinds: Pretrained User Tokens and Embeddings for User Understanding in Large Recommender Systems
YouTube 团队提出工业级系统 TokenMinds，将 PLUM 框架从基于 SID 的物品检索扩展到生成式序列用户建模。该系统采用改编自预训练 LLM 的编码器-解码器架构，编码器输出整体用户状态的稠密嵌入，解码器生成多面向未来兴趣的 SID 用户 token，并用共享 SID 词表统一长短视频行为，将上游训练与推理计算分别减少 50% 和 31%。
Authors:Qingyun Liu, Bo Yan, Yang Liu, Yuji Roh, Ekansh Sharma, Likang Yin, Emma Olowo, Yuxuan Li, Yikun Jiang, Diego Uribe, Saksham Aggarwal, Siqi Wu, Yuan Hao, Vikas Kedigehalli, Min-hsuan Tsai, Lukasz Heldt, Lichan Hong, Li Wei, Xinyang Yi
Abstract:User modeling in industrial recommender systems typically produces dense embeddings, which suffer from representational constraints due to fixed-dimensional vectors. As an emerging alternative for discrete user representation, LLM-generated natural-language user profiles often capture language semantics rather than sequential patterns in user behaviors. Meanwhile, Semantic ID (SID) is effective for item representation, yet existing discrete user representations remain largely confined to quantizing already-pooled user vectors, clustering historical watches, or predicting coarse topic clusters solely for candidate retrieval. We propose TokenMinds, an industrial system that extends the PLUM framework from SID-based item retrieval to generative sequential user modeling. Built on an encoder-decoder architecture adapted from pretrained LLMs, TokenMinds introduces a dual-output design: the encoder outputs a dense user embedding capturing a holistic user state, while the decoder generates discrete SID-based user tokens representing multi-faceted future interests. Grounded in item semantics, these tokens preserve distinct interests that a single pooled vector would average away. Additionally, a shared SID vocabulary across long- and short-form videos unifies cross-scenario behaviors into a single model, cutting upstream training and serving compute by 50% and 31%. Furthermore, leveraging the text vocabulary inherited from the pretrained LLM, TokenMinds interleaves textual search queries directly with watch histories to capture explicit user intent. We validate TokenMinds through offline evaluations and live A/B experiments across YouTube's primary ranking surfaces, serving billions of users asynchronously: SID-based user tokens are effective on their own, yield additive gains alongside dense embeddings, and together outperform the previous-generation, non-LLM user model in production.
| Subjects: | Information Retrieval (cs.IR); Artificial Intelligence (cs.AI); Machine Learning (cs.LG) |
| Cite as: | arXiv:2606.25147 [cs.IR] |
| (or arXiv:2606.25147v2 [cs.IR] for this version) | |
| https://doi.org/10.48550/arXiv.2606.25147 arXiv-issued DOI via DataCite |
Submission history
From: Qingyun Liu [view email]
[v1]
Tue, 23 Jun 2026 20:20:37 UTC (1,384 KB)
[v2]
Mon, 26 Oct 2026 22:51:11 UTC (1,130 KB)
来源：arXiv cs.AI · arxiv.org