跳到正文
arXiv cs.AI· Liang Su, David Chen, Jason Li, Kevin Guo, Teresa Lv·· 4 小时前AI 评分58

FLASHRT-PAI:面向物理 AI 的低延迟推理引擎论文发布

FlashRT-PAI: Fast Kernels Are Not Fast Pipelines for Physical AI

AI 导读

FLASHRT-PAI 论文提出以模型-硬件管线协同设计替代孤立 kernel 优化,解决 VLA、VA 与世界动作模型实时单批次推理延迟问题。系统通过深度 kernel 融合与启动重叠、执行契约和执行状态胶囊实现跨决策状态复用与快照恢复,并支持无改动 PyTorch 模型的 drop-in 加速。

正文 · 原文

View PDF HTML (experimental)

Abstract:Vision-language-action (VLA), video-action (VA), and world action models (WAMs) form the backbone of modern physical AI systems, yet their real-time single-batch inference remains a defining bottleneck. Latency is dominated not by individual kernel computation, but by system-level overhead: thousands of short kernel launches, intermediate data movement across operator boundaries, and redundant recomputation across consecutive decisions. General-purpose graph compilers cannot address these costs, as they optimize individual calls in isolation and lack visibility into cross-call state semantics and global pipeline structure. We present FLASHRT-PAI, a low-latency inference engine built around model-hardware pipeline co-design rather than isolated kernel optimization. FLASHRT-PAI combines three core mechanisms: deep kernel fusion with launch overlap eliminates inter-operator boundaries and launch gaps; an execution contract formalizes the pipeline's stage DAG, update rates, and named buffers to enable cross-decision state reuse and selective refresh; and execution-state capsules enable fast snapshot, restore, and fork of persistent pipeline state. The system also supports drop-in acceleration of unmodified PyTorch models via parity- and latency-gated kernel replacement. Evaluation covers six policies on RTX 5090 and four on Jetson AGX Thor. At matched 16-bit precision, FLASHRT-PAI is 1.3-3.7x faster than official compiled paths. Combining pipeline co-design with reduced precision delivers 3.38-12.66x speedup over official eager implementations. In LIBERO-10, four-bit pi0.5 achieves 94.04% success on Thor and 93.32% on RTX 5090, compared with 92.4% for the measured official FP16 reference. FLASHRT-PAI is open source, distributed via Hugging Face Kernel Hub, and its opt-in pi0.5 kernels have been adopted into the LeRobot framework.
Comments: source code this https URL
Subjects: Artificial Intelligence (cs.AI)
Cite as: arXiv:2610.24349 [cs.AI]
  (or arXiv:2610.24349v2 [cs.AI] for this version)
  https://doi.org/10.48550/arXiv.2610.24349

arXiv-issued DOI via DataCite

Submission history

From: Jason Li [view email]
[v1] Sat, 17 Oct 2026 00:35:58 UTC (71 KB)
[v2] Thu, 12 Nov 2026 23:30:40 UTC (63 KB)

来源:arXiv cs.AI · arxiv.org