MCSTN：面向低质量可穿戴传感器数据的人体活动识别缺失条件时空融合方法
Missingness-Conditioned Spatio-Temporal Fusion for Human Activity Recognition from Low-Quality Wearable Sensor Data
AI 导读
研究者提出 MCSTN，一种端到端框架，可在传感器通道缺失、噪声等不完美观测下直接进行人体活动识别，无需先插补再识别。该模型在 PAMAP2、Opportunity、WISDM 三个数据集上分别取得 94.24%、91.67%、96.22% 的平均 macro-F1，均为对比方法中最佳。参数量不足 100 万，单样本推理延迟 3.78ms。
正文 · 原文
Abstract:Body-worn sensor networks capture human activities through complementary views, where nodes, modalities, and axes partially describe the underlying motion. In real deployments, hardware faults, wireless dropouts, and loose attachments often cause entire channels or nodes to go missing, while environmental interference degrades the remaining signals. The common impute-then-recognize pipeline introduces reconstruction errors that cannot be observed or corrected at inference. We propose the Missingness-Conditioned Spatio-Temporal Network (MCSTN), an end-to-end framework for robust activity recognition under imperfect sensing. MCSTN has three components: a structured imperfection generator that simulates point, temporal-block, axis, modality, and node failures, plus noise at varying signal-to-noise levels; a mask-aware representation that conditions node embeddings, attention, and temporal aggregation on the availability mask; and a complete-to-imperfect multi-view learning scheme where an exponential-moving-average teacher guides multiple degraded student views through semantic consistency. The encoder models temporal and cross-feature dependencies in parallel: a bi-directional selective state-space branch captures long-range temporal dependencies, a transposed-attention branch models cross-channel correlations, and adaptive weights fuse their outputs. At inference, the generator, teacher, and consistency loss are removed, so MCSTN predicts directly from the observed window and its mask without reconstruction or noise-level estimation. On PAMAP2, Opportunity, and WISDM, MCSTN achieves average macro-F1 scores of 94.24\%, 91.67\%, and 96.22\% under imperfect observations, the best among all compared methods on each dataset, and the best in every failure mode on Opportunity. It requires fewer than one million parameters, with 3.78~ms inference latency per sample.
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2605.00913 [cs.CV] |
| (or arXiv:2605.00913v2 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2605.00913 arXiv-issued DOI via DataCite |
Submission history
From: Jiangtao Fan [view email]
[v1]
Wed, 29 Apr 2026 10:55:41 UTC (9,626 KB)
[v2]
Mon, 26 Oct 2026 12:57:05 UTC (7,925 KB)
来源：arXiv cs.AI · arxiv.org