跳到正文
arXiv cs.AI· Mizuho Okitani, Yoshinobu Hagiwara·· 4 小时前AI 评分31

面向变化观察条件的多模态人物概念模型用于人物识别

Multimodal Person Concept Model for Person Identification under Varying Observation Conditions

AI 导读

一项研究提出无需人物标签的多模态人物概念模型,基于中国餐馆过程的非参数贝叶斯生成模型,融合 CLIP 与 CLAP 提取的视觉、听觉和语言特征并经 PCA 处理,从少量观察中学习人物概念。

正文 · 原文

View PDF HTML (experimental)

Abstract:Service robots in domestic and office environments must identify users to provide personalized services based on their preferences and previous interactions. This is challenging because only a few observations may be available for learning, while observation conditions change owing to appearance, environmental noise, and utterance content. To address this challenge, this study proposes a multimodal person concept model that learns person concepts from a small number of observations, without person labels, and enables robust identification under changing conditions. The model employs a nonparametric Bayesian generative model based on the Chinese restaurant process and integrates visual, acoustic, and linguistic features extracted using Contrastive Language-Image Pre-training (CLIP) and Contrastive Language-Audio Pretraining (CLAP), followed by principal component analysis (PCA). We evaluated six participants under eight conditions in which appearance changes, additive noise, and transcription errors were introduced individually and in combination. Averaged over the eight conditions, the proposed model achieved an identification accuracy of 0.89 while inferring the number of person concepts from the observations, which is comparable to k-means clustering that was supplied with the true number of participants (0.88). We also examined PCA dimensionality, modality availability, and variation strength, and demonstrated the model on a Toyota Human Support Robot. These results show that multimodal person concept learning supports robust person identification under changing conditions, and that identification can succeed even when the number of concepts is not correctly recovered. The project page and demonstration video are available at this https URL.
Comments: 12 pages, 4 figures, 11 tables. Submitted to Artificial Life and Robotics (AROB Journal Track). Project page: this https URL
Subjects: Robotics (cs.RO); Artificial Intelligence (cs.AI); Human-Computer Interaction (cs.HC); Machine Learning (cs.LG)
Cite as: arXiv:2611.17802 [cs.RO]
  (or arXiv:2611.17802v1 [cs.RO] for this version)
  https://doi.org/10.48550/arXiv.2611.17802

arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Mizuho Okitani [view email]
[v1] Fri, 13 Nov 2026 06:30:53 UTC (4,150 KB)

来源:arXiv cs.AI · arxiv.org