从可复用性与可验证性评估思维链推理传递
Evaluating Chain-of-Thought Reasoning through Reusability and Verifiability
AI 导读
一项 arXiv 研究用 Sender-Receiver 双 LLM 实验评估模型间思维链(Chain-of-Thought)推理传递,提出可验证性、可纠正性、欺骗性三项指标,分别衡量传递的可靠性、有用性与安全性。结果显示任务类型对可靠性影响最大——算术任务可靠性高于科学与常识任务;接收方特性主要决定有用性,发送方特性则最影响错误推理对接收方的危害。
正文 · 原文
Abstract:AI applications increasingly compose multiple models, where one model acts on the output produced by another model. The transferred text can also carry the reasoning of the sender model. Chain-of-thought is our experimental instance of such text. We ask three questions about such inter-model reasoning transfer. Is the reasoning transmitted reliably? Is it useful to the receiver? Is it safe? We study these three questions with a Sender-Receiver setup. A Sender LLM produces a reasoning trace. A separate Receiver LLM must act on it. Neither model knows the identity of the other. The Sender therefore cannot tailor its reasoning to a particular Receiver. Reliability asks whether the Receiver recovers the Sender's intended conclusion. We measure it with Verifiability. Usefulness asks whether transferred reasoning helps a Receiver correct its mistakes. We measure it with Correctability. Safety asks whether transferred reasoning makes a Receiver abandon a correct answer and follow the wrong reasoning supplied by the Sender. We measure it with Deceptiveness. All three factors (Task, Sender, and Receiver) affect all three metrics that we measure. However, there are some interesting trends. Nature of task is a more important factor in the reliability of reasoning transfer. Reliability is higher on arithmetic tasks and lower on science and commonsense tasks. The nature of Receiver plays more important role while deciding the usefulness of reasoning transfer. The nature of the Sender matters most for whether corrupted reasoning harms the Receiver.
| Subjects: | Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Information Retrieval (cs.IR) |
| Cite as: | arXiv:2602.17544 [cs.AI] |
| (or arXiv:2602.17544v2 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2602.17544 arXiv-issued DOI via DataCite |
Submission history
From: Shashank Aggarwal [view email]
[v1]
Thu, 19 Feb 2026 16:59:11 UTC (4,747 KB)
[v2]
Fri, 13 Nov 2026 14:57:37 UTC (67 KB)
来源:arXiv cs.AI · arxiv.org