EvoSpeakerMem：面向多用户语音智能体的在线说话人-记忆构建
From Voiceprints to User Entities: Online Speaker-Memory Construction for Multi-User Spoken Agents
AI 导读
论文提出 EvoSpeakerMem，一个免训练的说话人路由与记忆构建层，将声纹视为身份观察而非身份本身，支持因果说话人路由、选择性声学状态更新和按身份分区的记忆读写。
正文 · 原文
Abstract:Personalized spoken agents must identify speakers before accessing user-specific memory, yet memory benchmarks typically assume correctly attributed histories. We argue that voiceprints are observations of identity, not identities themselves, and formulate online speaker-memory construction to maintain persistent user entities as acoustic evidence evolves. We introduce EvoSpeakerMem, a training-free layer for causal speaker routing, selective acoustic-state updates, and identity-partitioned memory writes and retrieval through a replaceable text-memory backend. We also construct LoCoMo-ID, a spoken-memory benchmark with hidden speaker identities, speaker-sensitive queries, simulated recording-condition shifts, and four-speaker identity-confusion tests. On longitudinal SMIIP-TV, genuine-probe rejection under a frozen near-enrollment threshold rises from 29.8% to 55.6% over approximately one year. On LoCoMo-ID speaker-sensitive queries, gated updating improves identity-gated QA over fixed representations by 3.86 and 11.62 percentage points under clean and shifted conditions, respectively, with no wrong-user authorizations. The advantage persists in the four-speaker test. Yet reducing fragmentation and improving QA can coexist with incorrect cross-user links, revealing a fragmentation--contamination trade-off that QA alone cannot capture.
| Comments: | Preprint |
| Subjects: | Sound (cs.SD); Artificial Intelligence (cs.AI); Audio and Speech Processing (eess.AS) |
| Cite as: | arXiv:2610.36713 [cs.SD] |
| (or arXiv:2610.36713v1 [cs.SD] for this version) | |
| https://doi.org/10.48550/arXiv.2610.36713 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Yucong Zhang [view email]
[v1]
Tue, 27 Oct 2026 06:27:24 UTC (1,641 KB)
来源：arXiv cs.AI · arxiv.org