arXiv cs.AI· Xiaodong Wu, Zhimin Zhao, Qi Li, Chaklam Cheong, Xiangman Li, Jianbing Ni·· 4 小时前AI 评分49
研究评测 20 家 LLM API 中转服务：GPT-6 Astra 与 Claude Opus 5 的回答行为、工具权限与计费问题
What the Middleman Delivers: Answer Behavior, Tool Permissions, and Billed Usage in LLM API Relays
AI 导读
一项研究测试了 20 家提供 GPT-6 Astra 和 Claude Opus 5 的第三方 API 中转服务。首轮样本中有 7/10 低价中转和 2/10 同价或更高价中转被发现回答行为差异；四家中转在指令禁用工具调用的情况下仍返回工具调用；五家对相同请求多报 4,200 至 60,000 个 input tokens。研究表明仅看宣传的模型和价格不足以判断中转服务可靠性，需多次重复测量。
正文 · 原文
Abstract:Third-party application programming interface (API) relays offer access to large language models (LLMs), often at lower prices. Users depend on these services for model quality, instruction compliance, and accurate billing. However, existing studies leave unclear whether relays meet these expectations together. We test 20 relays offering GPT-6 Astra and Claude Opus 5. First, we compare answers to knowledge questions with batch-matched official responses. In our first-round sample, the analysis flags 7/10 low-price and 2/10 same- or higher-price relays for behavioral differences. On retesting 35 pairs, seven gain or lose a flag under the original rules, and six under matched correction families. Because answer comparisons cannot reveal every delivery problem, we examine returned content and tool permissions. No injection is confirmed in 3,332 screened responses to a fixed, non-adversarial code and setup workload, yet four relays return tool calls despite instructions disabling them in the first round. Finally, to connect reported usage to actual costs, we compare input-token counts with official references and check balances. For identical requests, five relays report 4,200 to 60,000 more input tokens than the official API. Where balances are available (six services), charges follow reported usage at listed prices, including extra reported input tokens at one service. Our study shows that advertised models and prices are insufficient to judge relay reliability. It identifies unexplained billed token quantities and tool-permission violations as risks and shows why service evaluations need repeated measurements.
| Subjects: | Software Engineering (cs.SE); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2611.07650 [cs.SE] |
| (or arXiv:2611.07650v1 [cs.SE] for this version) | |
| https://doi.org/10.48550/arXiv.2611.07650 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Xiaodong Wu [view email]
[v1]
Thu, 5 Nov 2026 08:35:38 UTC (224 KB)
来源：arXiv cs.AI · arxiv.org