ReferClaw：消解自我中心辅助场景中的用户意图歧义
ReferClaw: Disambiguating User Intent for Egocentric Assistance
AI 导读
arXiv 论文提出 ReferClaw，一个面向自我中心流式问答的智能体框架，在回答前通过提问澄清用户意图。框架由三个协作智能体识别候选解释、请求澄清并基于意图与视觉证据生成答案，配合有界的成本感知策略限制澄清轮次，并通过意图相关技能更新与过程奖励引导的策略优化从交互中学习。
正文 · 原文
Abstract:Users of egocentric AI assistants often ask short, ambiguous questions whose intended referents cannot be uniquely resolved from video and dialogue history. For example, "Where did I leave it?" may refer to several previously handled objects. Existing VideoQA systems typically commit to a single interpretation, risking plausible but intention-misaligned answers. We introduce ReferClaw, an agentic framework that formulates egocentric streaming QA as an interactive process: the assistant asks clarifying questions to resolve user intent before answering. Three cooperating agents identify candidate interpretations, request clarification, and generate answers grounded in the resolved intent and visual evidence. A bounded, cost-aware policy balances expected information gain against user effort while limiting clarification turns and question complexity. Referclaw learns from interaction through intent-specific skill updates and process-reward-guided policy optimization with agent-level credit assignment. Evaluations on augmented egocentric QA benchmarks demonstrate improved answer accuracy and joint answer-and-grounding performance over existing baselines, alongside zero-shot transfer across datasets. These findings support interactive clarification as a practical approach to resolving online user intent in egocentric streaming assistance.
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Multiagent Systems (cs.MA) |
| Cite as: | arXiv:2611.07683 [cs.CV] |
| (or arXiv:2611.07683v1 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2611.07683 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Hangyu Qin [view email]
[v1]
Thu, 5 Nov 2026 09:10:14 UTC (1,123 KB)
来源：arXiv cs.AI · arxiv.org