arXiv cs.AI· Huajian Zeng, Chaohua Yao, Yuantai Zhang, Jiaqi Yang, Rolandos Alexandros Potamias, Xingxing Zuo·· 4 小时前AI 评分41
StableHand：质量感知 flow matching 实现从第一视角视频估计世界坐标系双手运动
StableHand: Quality-Aware Flow Matching for World-Space Dual-Hand Motion Estimation from Egocentric Video
AI 导读
StableHand 是一个质量感知 flow matching 框架，用于从第一视角视频恢复世界坐标系下双手交互的 4D 运动，由学习到的质量网络预测四通道质量信号，并通过逐通道前向调度、质量调整的速度目标、DiT 去噪器的 AdaLN 调制和质量感知 ODE 初始化融入生成过程。
正文 · 原文
Abstract:Recovering world space 4D motion of two interacting hands from egocentric video is a fundamental capability for supervising robot policy learning, where wrist trajectories track the end-effector and finger articulations specify the grasp pose. Two major challenges arise in this setting: hands frequently leave the camera view for extended periods due to head motion, and persistent hand-object interactions cause severe occlusions of one or both hands. Existing methods uniformly condition on noisy hand motion observations without accounting for their per-frame reliability, leading to substantial performance degradation. Our key insight is that accurate world space hand motion estimation is tightly coupled with the quality of per-frame hand observations. To this end, we decompose the quality of hand motion observations extracted from an off-the-shelf hand pose estimator into four channels: wrist global translation and finger articulations for both hands. We propose StableHand, a quality-aware flow-matching framework conditioned on these four-channel quality signals, which are predicted by a learned quality network. We naturally incorporate the quality signals into the flow-matching process through a per-channel forward schedule, a quality-adjusted velocity target, AdaLN modulation of the DiT denoiser, and a quality-aware ODE initialization. This unified generative process preserves high-quality observations while reconstructing unreliable ones using a learned bimanual motion prior. Experiments on HOT3D and ARCTIC, two egocentric benchmarks featuring long missing-hand spans and persistent hand-object occlusions, show that StableHand achieves state-of-the-art performance across all reported metrics, reducing W-MPJPE by 20-25% compared to the strongest baseline, with the largest gains on heavily occluded ARCTIC sequences.
| Comments: | Accepted by NeurIPS 2026. Project Page: this https URL |
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2605.18553 [cs.CV] |
| (or arXiv:2605.18553v2 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2605.18553 arXiv-issued DOI via DataCite |
Submission history
From: Huajian Zeng [view email]
[v1]
Mon, 18 May 2026 15:33:15 UTC (5,503 KB)
[v2]
Mon, 26 Oct 2026 14:05:12 UTC (4,893 KB)
来源：arXiv cs.AI · arxiv.org