跳到正文
arXiv cs.AI· Jingyu Tang, Xu Yang, Zhehao Zhang, Zheng Wang, Qi Cheng, Haozhe Zhao, Manling Li, Wenxuan Li, Xide Xia, Jiaxin Pei·· 2 小时前AI 评分46

Realtime-Bench:前沿模型能解决实时交互问题吗?

Realtime-Bench: Can Frontier Models Solve Real-time Interactive Problems?

AI 导读

研究者推出 Realtime-Bench 基准,用动态任务测试实时流式模型在时间约束下行动的能力。在环境按实际推理延迟推进的 Wall Clock 模式下,最佳成功率仅 44%,最佳 Pass^3 仅 36%。论文还引入 Paused Clock 和 Token Clock 两种计时方式分离服务延迟与时间感知,发现时间成本限制演进环境中的成功率,更高推理努力在时间压力下效果好坏参半。

正文 · 原文

View PDF HTML (experimental)

Abstract:Real-time interaction requires agents to perceive evolving environments and act under temporal constraints, yet existing evaluations of real-time streaming models focus on conversational quality and streaming perception, leaving effective action under strict temporal constraints untested. We introduce Realtime-Bench, a suite of carefully curated dynamic tasks where delayed actions directly affect success. Under Wall Clock, where environments advance according to actual inference latency, the best success rate reaches only 44% and the best Pass^3 is only 36%, demonstrating that real-time interaction remains challenging. To separate serving latency from models' real-time streaming capability, we compare Wall Clock with Paused Clock, which removes inference delays as an idealized reference, and introduce Token Clock, which advances simulated time according to generated token counts, testing temporal awareness independently of physical serving speed. We find that temporal costs limit success in evolving environments and that higher reasoning effort has mixed effects under time pressure, reflecting a trade-off between decision quality and timely execution. Based on these results, we discuss future directions for improving real-time multimodal models, including adaptive reasoning with temporal awareness and continuous perception and action.
Subjects: Artificial Intelligence (cs.AI)
Cite as: arXiv:2610.22155 [cs.AI]
  (or arXiv:2610.22155v4 [cs.AI] for this version)
  https://doi.org/10.48550/arXiv.2610.22155

arXiv-issued DOI via DataCite

Submission history

From: Jingyu Tang [view email]
[v1] Thu, 15 Oct 2026 16:20:47 UTC (5,954 KB)
[v2] Sun, 18 Oct 2026 17:26:02 UTC (5,954 KB)
[v3] Thu, 22 Oct 2026 00:38:01 UTC (4,925 KB)
[v4] Mon, 16 Nov 2026 21:03:58 UTC (2,254 KB)

来源:arXiv cs.AI · arxiv.org