arXiv cs.AI· Kevin Avdiu, Seongjin Bien, Yannik Blei, Philine Meister, Johannes Hechtl, Philipp Schmitt, Kai Wurm, Georg von Wichert, Wolfram Burgard·· 3 小时前AI 评分34
RUMI：匹配 Robotiq 2F-85 的通用操作数据采集接口，结合增强现实与触觉反馈
RUMI: A Robotiq-Matched Universal Manipulation Interface for Robot Learning with Augmented Reality and Haptic Feedback
AI 导读
研究者提出 RUMI，一种与 Robotiq 2F-85 夹爪几何匹配的手持通用操作接口，可更换指尖传感器，在接触前提供自然输入、接触后测量力。系统将追踪集成到 VR 头显中提供实时增强现实反馈，并引入触觉反馈机制支持力感知遥操作。用户研究显示，有增强现实时关节极限违规片段为 3/50，无增强现实时为 16/50。
正文 · 原文
Abstract:Many manipulation tasks require carefully controlled contact forces for successful execution, yet large-scale, force-aware datasets remain scarce. Handheld interfaces can help by transmitting interaction forces directly through the operator's arm, but most existing designs are tailored to simple linear grippers and do not match the nonlinear mechanics of widely used robotic grippers. We present RUMI, a handheld universal manipulation interface matched to the Robotiq 2F-85 with exchangeable fingertip sensors that preserves the target gripper geometry while enabling natural input before contact and force measurement after contact. By integrating the tracking system into a virtual reality headset, our method provides real-time augmented reality feedback about the kinematic feasibility of motion on the target robot platform. We further introduce a haptic feedback mechanism that supports force-aware teleoperation for simulated and real robots. A user study shows fewer episodes containing joint-limit violations with live robot visualization and pose initialization (3/50) than without augmented reality (16/50). We additionally train imitation policies from demonstrations, collected using two different fingertip modalities enabled by the gripper-matched hardware, and assess the haptic interface's capability for collecting force-aware teleoperation datasets in simulation.
| Subjects: | Robotics (cs.RO); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2611.15802 [cs.RO] |
| (or arXiv:2611.15802v1 [cs.RO] for this version) | |
| https://doi.org/10.48550/arXiv.2611.15802 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Yannik Blei [view email]
[v1]
Wed, 11 Nov 2026 18:17:42 UTC (4,299 KB)
来源：arXiv cs.AI · arxiv.org