arXiv cs.AI· Dong Bo, Fanjin Zhang, Ke Cheng, Xinxin Zhao, Xiaowei Su, Yang Wang, Jinghang Xu, Cuiping Li, Hong Chen, Jing Zhang·· 2 小时前AI 评分36
BuySideFlow：面向投资研究 LLM Agent 的时点（PIT）基准
BuySideFlow: A Point-in-Time Benchmark for LLM Agents in Investment Research
AI 导读
BuySideFlow 是一个投资研究 LLM 智能体基准，包含 408 个源自从业者的中文任务，覆盖股票、基金、固收与宏观领域，数据来自持牌生产级金融数据库的固定快照。九种 agent 配置的结果准确率为 16.2% 至 36.8%；对 100 个共享任务共 900 次运行的审计显示，通过结果检查的 380 次运行中有 193 次（50.8%）存在时点违规，表明结果一致不等于时点有效。
正文 · 原文
Abstract:Large language models (LLMs) can generate SQL and SQL--Python workflows for investment research, but ensuring point-in-time (PIT) validity remains challenging. Agents must enforce disclosure cutoffs and select the appropriate historical states and record versions throughout data retrieval and financial analysis. We introduce \textsc{BuySideFlow}, a benchmark of 408 practitioner-derived Chinese-language tasks spanning equities, funds, fixed income, and macroeconomics over a fixed snapshot from a licensed production financial database. We provide executable references and a taxonomy of temporal errors, pairing result checks with expert audits of executed workflows. We evaluate nine agent configurations on all tasks, with result accuracy ranging from 16.2\% to 36.8\%. An audit of 900 runs on 100 shared tasks confirms PIT violations in 193 of the 380 runs that pass result checks (50.8\%), showing that output agreement alone does not establish temporal validity.
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2611.12966 [cs.AI] |
| (or arXiv:2611.12966v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2611.12966 arXiv-issued DOI via DataCite |
Submission history
From: Dong Bo [view email]
[v1]
Sat, 7 Nov 2026 12:55:19 UTC (502 KB)
来源：arXiv cs.AI · arxiv.org