arXiv cs.AI· Shuhan Li, Huankang Guan, Xintong Guo, Yang Bo, Yuyang He, Dunyuan Xu, Jingyan Chen, Fang Liu, Tsz Kin Wan, Jinpeng Chen, Zhenqi He, Yuxuan Hu, Wenbo Li, Han Gao, Qijun Li, Long Chen, Hongsheng Li, Rui Liu·· 3 小时前AI 评分45
AURA 2.0：从用户驱动响应走向流式视频中的模型主动发起对话
AURA 2.0: From User-Driven Responses to Model-Initiated Conversations in Streaming Video
AI 导读
AURA 2.0 是一款流式视频大语言模型，在 AURA 基础上引入三大升级：模型主动发起对话（MIC）、流感知工具调用（SATC）和交互式视觉定位（IVG），并可输出帧对齐的边界框。模型通过专用流式交互数据集和三阶段训练开发新能力，并加入长时交互记忆（LHIM）保持长视频上下文连续性。该模型在公开流式基准上达到开源流式模型的领先水平，模型权重与推理代码已开源。
正文 · 原文
Authors:Shuhan Li, Huankang Guan, Xintong Guo, Yang Bo, Yuyang He, Dunyuan Xu, Jingyan Chen, Fang Liu, Tsz Kin Wan, Jinpeng Chen, Zhenqi He, Yuxuan Hu, Wenbo Li, Han Gao, Qijun Li, Long Chen, Hongsheng Li, Rui Liu
Abstract:Most AI assistants respond only when prompted, remaining silent even when events in the video stream offer meaningful opportunities for conversation. To move beyond purely user-driven interaction, we present AURA 2.0, a streaming video large language model that introduces three core upgrades to AURA. The key advance is Model-Initiated Conversation (MIC), which enables the model to identify topics worth discussing in the evolving visual stream and initiate conversations at appropriate moments without a user query. Stream-Aware Tool Calling (SATC) enables the model to invoke external tools using the current visual and conversational context and incorporate returned information into the ongoing interaction. Interactive Visual Grounding (IVG) supports visual grounding in both user inputs and model outputs. The model can identify objects marked by users on video frames and produce frame-aligned bounding boxes to link its responses to specific objects and regions. To develop these capabilities, we construct a dedicated streaming interaction dataset and adopt a three-stage training scheme to integrate the new behaviors with AURA's existing streaming interaction abilities. We further incorporate Long-Horizon Interaction Memory (LHIM) to maintain contextual continuity over extended video streams. Evaluations on public streaming benchmarks show that AURA 2.0 achieves leading performance among open-source streaming models, while targeted evaluations further validate its new interaction capabilities. We open-source the model weights and inference code.
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2611.17048 [cs.CV] |
| (or arXiv:2611.17048v1 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2611.17048 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Shuhan Li [view email]
[v1]
Thu, 12 Nov 2026 16:31:54 UTC (16,306 KB)
来源：arXiv cs.AI · arxiv.org