StyleInstructCaps：面向音频语言模型可控说话风格描述的首个标准化基准

View PDF HTML (experimental)

Abstract:Speaking-style captioning (SSC) aims to generate natural language descriptions of how speech is delivered, capturing paralinguistic attributes such as vocal timbre, prosody, and expressivity. Many downstream applications, including conversational AI agents, controllable speech generation, and large-scale audio annotation, require dimension-specific style captions that describe targeted aspects of speech (e.g., speaker traits, emotion, or delivery style), rather than a single undifferentiated description. However, existing work lacks a unified formulation for controllable speaking-style captioning. We address this gap by formulating SSC as an instruction-following audio-language task, where explicit instructions specify the speaking-style dimensions to be described. We introduce StyleInstructCaps, the first standardized benchmark for controllable speaking-style captioning in audio-language models. StyleInstructCaps provides a dataset and evaluation framework that measures metadata groundedness, hallucination, instruction-following ability, generalization to unseen instructions and audio datasets, and speaker-style consistency. Across nine state-of-the-art audio-language models, we find that strong instruction-following ability does not necessarily imply grounded speaking-style understanding, and that SSC-specific instruction-diverse training substantially improves groundedness, controllability, and speaker consistency.

Comments:	Accepted at AACL 2026. Code, data, and benchmark are available at this https URL
Subjects:	Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Sound (cs.SD); Audio and Speech Processing (eess.AS)
Cite as:	arXiv:2610.33036 [cs.CL]
	(or arXiv:2610.33036v1 [cs.CL] for this version)
	https://doi.org/10.48550/arXiv.2610.33036 arXiv-issued DOI via DataCite

Submission history

From: Shreeram Suresh Chandra [view email]
[v1] Thu, 15 Oct 2026 00:17:09 UTC (4,125 KB)