基于视觉语言的移动机器人语义导航：从自然语言到导航目标
From Language to Navigation Goals: A Vision-Language Approach for Semantic Navigation of Mobile Robots Using RGB-D Perception
AI 导读
一项arXiv论文提出语言驱动的移动机器人导航框架，可将"去邮箱"等自然语言指令通过模块化 ROS 2 组件转化为导航目标，用 RGB-D 数据估计目标位置并经 Nav2 导航栈执行。系统在仿真和真实场景中用 TurtleBot3 Waffle 和搭载 RealSense 相机的 Unitree Go2 验证，能理解直接命令与上下文请求并生成自然语言反馈，代码将在论文接收后开源。
正文 · 原文
Abstract:Natural language interaction provides an intuitive way for non-expert users to communicate with robotic platforms. However, transforming user requests into executable navigation actions remains a challenging task, requiring the integration of language understanding, environment perception, and autonomous navigation. This work presents a language-driven navigation framework that enables mobile robots to interpret user requests in natural language to move the robot to a destination and autonomously navigate towards it.
The framework is composed of modular ROS 2 components that cooperate to transform natural language instructions into navigation actions. Given a natural language request referring to a target in the environment (e.g., "go to the mail box"), the system identifies the referenced object, estimates its position using RGB-D data, and generates a navigation goal, which is then executed through the ROS 2 Nav2 navigation stack. The ROS 2-based implementation facilitates portability across different robotic platforms, requiring only the configuration of the corresponding topics and services.
The system is evaluated in both simulation and real-world scenarios using a TurtleBot3 Waffle and a Unitree Go2 robot with a RealSense camera. Experimental results show that the framework successfully interprets both direct commands and contextual requests, generates meaningful natural-language feedback, and navigates towards the desired target. These results demonstrate the feasibility of combining semantic perception and autonomous navigation to provide an intuitive human-robot interaction paradigm. Code will be released as open source upon acceptance.
| Comments: | 7 pages, 7 figures, 4 tables |
| Subjects: | Robotics (cs.RO); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2607.13624 [cs.RO] |
| (or arXiv:2607.13624v2 [cs.RO] for this version) | |
| https://doi.org/10.48550/arXiv.2607.13624 arXiv-issued DOI via DataCite |
Submission history
From: Jose Martínez-Fajardo [view email]
[v1]
Wed, 15 Jul 2026 09:15:21 UTC (1,720 KB)
[v2]
Tue, 27 Oct 2026 14:34:48 UTC (1,435 KB)
来源：arXiv cs.AI · arxiv.org