集成 VLM 的室内移动机器人语义自主框架：混合确定性推理与跨机器人自适应记忆

View PDF

Abstract:Semantic indoor navigation requires robots to connect human instructions with spatial destinations, execute missions and retain useful operational experience. This paper presents the Semantic Autonomy Stack (SAS), a six-layer integration framework, and develops its semantic-reasoning, mission-interface and operational-memory layers over a previously validated ROS 2/Nav2 foundation. A deterministic-first cascade combines the route graph, semantic objects, robot state and stored instruction-destination associations, escalating unresolved or guarded cases to a pretrained Vision-Language Model (VLM). Recurrent model-mediated associations can pass a frequency-and-consistency gate and be compiled for deterministic M3 reuse; this gate measures sampled-output stability rather than semantic truth. The frozen v4.8-review implementation was evaluated on two physical robot configurations in one indoor environment, with onboard navigation and perception and shared-workstation reasoning and memory. Of 216 decision cycles outside the start gate, 112 resolved deterministically at sub-millisecond resolver-compute latency. A sealed 28-instruction challenge yielded 17 acceptable outcomes; five fast-path decisions included one incorrect selection. One promoted association was reused on the second platform through deliberate shared-digest installation. A controlled 594-observation offline comparison showed greater agreement with the stored associations and lower decision latency from M3 reuse in five familiar cases. The M3-disabled cascade reduced VLM calls but matched direct-VLM decision quality in standardized context. Large latency savings were concentrated in paths avoiding VLM inference. Together, the results establish a working integration of selective hybrid reasoning and reusable operational memory, while identifying implementation and evaluation boundaries for further development within SAS.

Comments:	Published version (open access, CC BY 4.0). 32 pages, 6 figures, 10 tables, 2 algorithms. This version replaces the submitted manuscript (v1); the analysis, results and claims were revised during peer review, and the published article supersedes v1
Subjects:	Robotics (cs.RO); Artificial Intelligence (cs.AI)
Cite as:	arXiv:2605.02525 [cs.RO]
	(or arXiv:2605.02525v2 [cs.RO] for this version)
	https://doi.org/10.48550/arXiv.2605.02525 arXiv-issued DOI via DataCite
Journal reference:	Robotics and Autonomous Systems 207 (2027) 105822
Related DOI:	https://doi.org/10.1016/j.robot.2026.105822 DOI(s) linking to related resources

Submission history

From: Bogdan Felician Abaza [view email]
[v1] Mon, 4 May 2026 12:27:03 UTC (2,134 KB)
[v2] Sat, 24 Oct 2026 05:45:48 UTC (9,899 KB)