arXiv cs.AI· Tsunehiko Tanaka, Matthew Stephenson, Alistair Macvicar, Edgar Simo-Serra·· 5 小时前AI 评分32
基于视觉语言模型的证据门控任务与运动规划框架 EAFG
Evidence-Gated Task and Motion Planning with Vision-Language Models
AI 导读
研究者提出证据获取与可行性门控框架 EAFG，用 VLM 生成子目标、由 TAMP 规划动作序列获取视觉证据，再据此判断继续规划、补充证据或停止，以避免 VLM 子目标缺乏观测支持的问题。在涉及明确指定物品、指令欠指定和所需物品缺失的三种烹饪场景测试中，EAFG 在前两种场景下任务完成率优于 VLM-TAMP 基线，并能在物品缺失时正确停机。
正文 · 原文
Abstract:Robots executing long-horizon manipulation tasks from natural-language instructions must reason about both semantic task structure and geometric feasibility. Hierarchical approaches combining Vision-Language Models (VLMs) with Task and Motion Planning (TAMP) offer a promising solution, but VLM-generated subgoals may rely on prior knowledge without sufficient observational support. This can lead to the omission of task-relevant subgoals or attempts to execute instructions whose requirements cannot be met in the environment. We propose Evidence Acquisition and Feasibility Gating (EAFG), a framework for evidence-gated task and motion planning that acquires visual evidence to specify appropriate task plans and assess whether given instructions can be fulfilled. EAFG first acquires evidence by having a VLM generate subgoals and a robot execute action sequences planned by TAMP for these subgoals. For feasibility gating, the acquired evidence is used to decide whether to proceed with task planning, acquire further evidence, or halt. When proceeding, the VLM generates task subgoals conditioned on the acquired evidence, and TAMP computes low-level actions. We evaluate EAFG in three cooking scenarios involving explicitly specified required objects, underspecified instructions, and instructions requiring an absent object. EAFG improves task completion over a VLM-TAMP baseline in the explicit and underspecified scenarios and enables appropriate halting when a required object is absent.
| Subjects: | Robotics (cs.RO); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2608.20084 [cs.RO] |
| (or arXiv:2608.20084v2 [cs.RO] for this version) | |
| https://doi.org/10.48550/arXiv.2608.20084 arXiv-issued DOI via DataCite |
Submission history
From: Tsunehiko Tanaka [view email]
[v1]
Thu, 20 Aug 2026 14:17:52 UTC (620 KB)
[v2]
Thu, 29 Oct 2026 15:37:45 UTC (1,034 KB)
来源：arXiv cs.AI · arxiv.org