跳到正文
arXiv cs.AI· Guanzhong Sun, Junyi Ma, Yuxuan Wu, Erhang Zhang, Chenyang Xu, Yanzi Miao, Hesheng Wang·· 2 小时前AI 评分22

TOMAS:面向第一视角手部预测的拓扑感知操作建模与自适应分阶段框架

TOMAS: Topology-Aware Operation Modeling and Adaptive Staging for Egocentric Hand Forecasting

AI 导读

论文提出 TOMAS 框架,用于从第一视角视频预测未来手部运动。该框架包含拓扑感知的手-物体推理机制,将手部骨骼结构与物体语义关联,联合推断交互目标和潜在操作,并引入自适应分阶段机制,以可学习时间边界组织手部动作、用几何子目标引导轨迹细化。在多个公开基准上,TOMAS 在多维度、多目标手部运动预测中达到 SOTA。

正文 · 原文

View PDF HTML (experimental)

Abstract:Forecasting future hand movements from egocentric videos plays an important role in understanding human intentions and hand-object interactions. However, different manipulation intentions can lead to substantially different hand trajectories even when interacting with the same object. Existing methods have not sufficiently explored the intrinsic relationships between articulated hand movements and object semantics to capture such operation-dependent motion patterns. Therefore, we propose TOMAS, a topology-aware operation modeling and adaptive staging framework for egocentric hand forecasting. Specifically, we develop a topology-aware hand-object reasoning mechanism that associates hand skeletal structures and articulation dynamics with object semantics and spatial relationships to jointly infer potential interaction targets and latent manipulation operations. The resulting object-operation representations provide interaction conditions for future hand trajectory prediction, with supervision adapted to the annotations available in each dataset. Furthermore, we introduce an adaptive staging mechanism that organizes future hand movements into ordered latent stages with learnable temporal boundaries. Stage-specific geometric subgoals are then incorporated to guide trajectory refinement according to the underlying manipulation patterns. Experimental results on multiple publicly available benchmarks demonstrate that TOMAS achieves state-of-the-art performance in multi-dimensional and multi-target hand motion forecasting.
Subjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
Cite as: arXiv:2611.23970 [cs.CV]
  (or arXiv:2611.23970v1 [cs.CV] for this version)
  https://doi.org/10.48550/arXiv.2611.23970

arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Guanzhong Sun [view email]
[v1] Wed, 18 Nov 2026 07:10:44 UTC (38,145 KB)

来源:arXiv cs.AI · arxiv.org