跳到正文
arXiv cs.AI· Zora Zhiruo Wang, Apurva Gandhi, Rulin Shao, Aspen Chen, Jonas Mueller, Zhiqi Liang, Jett Chen, Michael Ryan, Qianou Ma, Luxi He, Zhoujun Cheng, Andre He, Seungone Kim, Jiayi Geng, Mingqian Zheng, Weiwei Sun, Zheyuan Zhang, Xinran Zhao, Yike Wang, Abe Hou, Liwei Jiang, Pang Wei Koh, Diyi Yang, Graham Neubig, Daniel Fried·· 4 小时前AI 评分52

TAHI:通过人机交互实现智能体的测试时个性化适应

Efficient Test-Time Adaptation through Human-Agent Interaction

AI 导读

卡内基梅隆等机构的研究者提出 TAHI,将跨会话人机交互信号融入智能体的上下文与权重,并通过演化的评分标准模块沉淀用户标准。在写作与视觉创作两个领域的 30 位用户、600 个任务上,智能体个体任务成功率提升 4.5-20.9%;评分标准模块生成的评估标准比大模型或人类单独生成的多捕捉 16.0-22.3% 的失败;个性化适应带来的改进还能跨用户泛化,成功率提升最高 8.8%。

正文 · 原文

Authors:Zora Zhiruo Wang, Apurva Gandhi, Rulin Shao, Aspen Chen, Jonas Mueller, Zhiqi Liang, Jett Chen, Michael Ryan, Qianou Ma, Luxi He, Zhoujun Cheng, Andre He, Seungone Kim, Jiayi Geng, Mingqian Zheng, Weiwei Sun, Zheyuan Zhang, Xinran Zhao, Yike Wang, Abe Hou, Liwei Jiang, Pang Wei Koh, Diyi Yang, Graham Neubig, Daniel Fried

View PDF HTML (experimental)

Abstract:AI agents are trained on population-scale data to encode broad capabilities spanning those of many practitioners. Yet the artifacts they produce rarely meet the personal bar professionals need to stake their reputation on. On realistic, open-ended tasks where success criteria are heterogeneous and insufficiently documented, individual expertise lives precisely in the elevation and departure from the average. In practice, iterative human-agent interaction surfaces criteria that users cannot fully specify up front, yet apply repeatedly across tasks. We argue this cross-session interaction data is a rich, underused signal for closing the gap to individual expertise. In this work, we propose test-time adaptation through human-agent interaction (TAHI), which integrates these signals into agent context and weights, and crystallizes each user's training and evaluation criteria via an evolving rubric module. We adapt agents to 30 individuals in two high-utility domains, writing and visual creation, on a total of 600 tasks. Our agents improve solo task success by 4.5-20.9% within only tens of tasks. Meanwhile, our evolving rubric module serves as a scalable annotation tool, creating evaluation rubrics that catch 16.0-22.3% more failures than those from LMs or humans alone. While agents are adapted towards individuals, we show these personalized agents also produce improvements in success of up to 8.8% that generalize across users.
Subjects: Artificial Intelligence (cs.AI)
Cite as: arXiv:2609.04141 [cs.AI]
  (or arXiv:2609.04141v2 [cs.AI] for this version)
  https://doi.org/10.48550/arXiv.2609.04141

arXiv-issued DOI via DataCite

Submission history

From: Zhiruo Wang [view email]
[v1] Thu, 3 Sep 2026 17:33:18 UTC (757 KB)
[v2] Fri, 13 Nov 2026 17:53:51 UTC (757 KB)

来源:arXiv cs.AI · arxiv.org