跳到正文
arXiv cs.AI· Zheng-Xin Yong, Benjamin Arnav, Miles Wang, Tomek Korbak, Olivia Watkins·· 56 分钟前精选AI 评分68

arXiv 论文揭示思维链可控性会削弱思维链可监控性

Chain-of-Thought Controllability Can Reduce Chain-of-Thought Monitorability

AI 导读

arXiv 论文(arXiv:2611.21799)通过 persona-control 训练压力测试思维链(CoT)监控风险,发现即使训练中不含逃避监控的样本,模型也会获得控制 CoT 的能力。

推荐理由

论文量化了思维链可控性对监控能力的削弱,并指出透明度提示可部分恢复监控,给出可复用的测试方法。

正文 · 原文

View PDF HTML (experimental)

Abstract:Chain-of-thought (CoT) monitoring is a promising tool for detecting hidden goals, off-task behavior, and other failures in AI agents. One risk is that models could deliberately shape their reasoning to make it opaque to monitors. We stress-test this risk through persona-control training: supervised fine-tuning on mathematical reasoning traces rewritten in the voices of diverse personas. This induces a broader ability to control CoT, even though training contains no monitor-evasion examples. In covert side-task evaluations, traces in which a gpt-oss-20b model organism completes the hidden side task contain less monitorable evidence than the base model's. Averaged across evaluations and CoT lengths, weighted by side-task-success counts, the CoT monitorability metric g-mean$^2$ falls from 0.94 for the base model to 0.68 for the model organism without a hiding instruction, and to 0.22 when explicitly instructed to hide. In intervention evaluations, explicit hiding often drives monitorability near zero. The effect extends to a Kimi-K2.6 model organism in agentic environments, though hidden objectives requiring extended reasoning or longer trajectories remain more monitorable than simple single-step side tasks. We further discover that targeted transparency prompts can substantially recover monitorability, sometimes surpassing the base model. These model organisms provide a testbed for studying monitorability failures and potential mitigations. Our code can be found on this https URL.
Comments: 22 pages, 17 figures
Subjects: Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
Cite as: arXiv:2611.21799 [cs.AI]
  (or arXiv:2611.21799v1 [cs.AI] for this version)
  https://doi.org/10.48550/arXiv.2611.21799

arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Zheng-Xin Yong [view email]
[v1] Mon, 16 Nov 2026 19:17:47 UTC (3,988 KB)

来源:arXiv cs.AI · arxiv.org