VoiceLongMemEval 基准：测试 AI 助手能否记住你说话时的声音特征
VoiceLongMemEval: Do Assistants Remember How You Sounded?
AI 导读
arXiv 论文提出 VoiceLongMemEval（VLME）基准，考察大语言模型在多轮长对话中能否利用副语言元数据（情绪标签、韵律描述、语音事件）回答问题，每题经三阶段对抗门槛确保仅凭文本无法作答。评测显示提供文本副语言元数据可带来 0.09 至 0.38 的准确率提升，音频原生模型可从语音直接提取线索（0.354 至 0.412 对比盲测的 0.325），而标准 ASR 流程会丢弃该信号。
正文 · 原文
Abstract:With the growing scale of multi-agent architectures and large language models, deployed AI assistants are increasingly tasked with reasoning over long, continuous, multi-session conversation histories. Current benchmarks evaluate this dialogue history as information retrieval over long horizon, temporal reasoning, or knowledge updates, while crucially ignoring the fundamental dynamics of human-agent interaction, i.e. how they said it. To address this gap, we present VoiceLongMemEval (VLME) benchmark, where every answer depends on paralinguistic metadata (emotion labels, prosody descriptors, and voice events) attached to conversational turns, which is otherwise unrecoverable from the words alone. Every item passes a three-stage adversarial gate, ensuring that a strong language model fails when given only the transcript. Evaluating leading frontier and open-weight models reveals a pervasive affect gap; providing text-track paralinguistic metadata yields a 0.09 to 0.38 accuracy boost (0.61 to 0.69 when prompted with evidence hints), while standard ASR pipelines systematically discard this signal. Additionally, audio-native models successfully extract these cues directly from speech (0.354 to 0.412 vs. 0.325 blind). Code and dataset will be made available upon acceptance.
| Subjects: | Artificial Intelligence (cs.AI); Machine Learning (cs.LG) |
| Cite as: | arXiv:2609.00570 [cs.AI] |
| (or arXiv:2609.00570v3 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2609.00570 arXiv-issued DOI via DataCite |
Submission history
From: Ramit Pahwa [view email]
[v1]
Tue, 1 Sep 2026 02:06:10 UTC (88 KB)
[v2]
Wed, 2 Sep 2026 03:32:07 UTC (80 KB)
[v3]
Thu, 29 Oct 2026 17:32:31 UTC (38 KB)
来源：arXiv cs.AI · arxiv.org