CFGComposer：基于 Copula 分数修正的音频-文本到动作组合式生成
CFGComposer: Compositional Audio-Text-to-Motion Generation via Copula Score Correction
AI 导读
arXiv 论文提出 CFGComposer，一个联合训练的 Audio-Text-to-Motion（AT2M）扩散模型，通过建模 copula 分数并以 CFG 类比方式注入 ε 空间修正，实现语音节奏与文本语义同步的全身动作生成。
正文 · 原文
Abstract:Generating whole-body gestures that are simultaneously rhythmically synchronized with speech audio and semantically aligned with text descriptions is a fundamental challenge in embodied AI and social robotics. We tackle this Audio-Text-to-Motion (AT2M) problem by composing two independently-trained diffusion models at inference time and refine bias in joint inference by modeling the copula score $s_I$. A rigorous theoretical analysis reveals that naive epsilon blending corresponds to a Product-of-Experts formulation that omits the copula score $s_I$: the cross-modal interaction between audio and text that neither marginal model can capture. We propose CFGComposer: a jointly-trained AT2M diffusion model that estimates $s_I$ via a CFG analogy and injects it as an $\epsilon$-space correction. Experiments on BEAT demonstrate best-in-class semantic fidelity (SRGR $0.9916$, FID $27.18$) with GT-level beat alignment (BA $0.1826$ vs. GT $0.1743$), and 76-83% win rate in blind pairwise human evaluation against all prior methods. Physical deployment on a Unitree G1 humanoid confirms direct sim-to-real transfer.
| Subjects: | Sound (cs.SD); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2611.15101 [cs.SD] |
| (or arXiv:2611.15101v1 [cs.SD] for this version) | |
| https://doi.org/10.48550/arXiv.2611.15101 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Pengyang Shi [view email]
[v1]
Wed, 11 Nov 2026 06:28:18 UTC (8,640 KB)
来源：arXiv cs.AI · arxiv.org