代码大语言模型隐私风险的因果视角：训练动态如何决定 PII 泄露
A Causal Perspective on the Role of Training Dynamics for Interpreting Privacy Risks in Code Models
AI 导读
一项 arXiv 研究首次从因果角度证实，代码大语言模型（LLM4Code）的 PII 泄露风险因信息类型而异。研究者构建含多样 PII 类型的数据集，微调不同规模的代表性模型，并用结构因果模型估计可学习性对泄露的因果效应。结果显示 IP 地址等易学样本泄露风险更高，密钥和密码等难学类型泄露较少，据此提出类型感知与可学习性感知的防御思路。
正文 · 原文
Abstract:Large language models for code (LLM4Code) have greatly improved developer productivity but also raise privacy concerns due to their reliance on open-source repositories containing abundant personally identifiable information (PII). Prior work shows that commercial models can reproduce sensitive PII, yet existing studies largely treat PII as a single category and overlook the heterogeneous risks among different types. We investigate whether distinct PII types vary in their likelihood of being learned and leaked by LLM4Code, and whether this relationship is causal. Our methodology includes building a dataset with diverse PII types, fine-tuning representative models of different scales, computing training dynamics on real PII data, and formulating a structural causal model to estimate the causal effect of learnability on leakage. Results show that leakage risks differ substantially across PII types and correlate with their training dynamics: easy-to-learn instances such as IP addresses exhibit higher leakage, while harder types such as keys and passwords leak less frequently. Ambiguous types show mixed behaviors. This work provides the first causal evidence that leakage risks are type-dependent and offers guidance for developing type-aware and learnability-aware defenses for LLM4Code.
| Comments: | 27 pages |
| Subjects: | Software Engineering (cs.SE); Artificial Intelligence (cs.AI); Cryptography and Security (cs.CR) |
| Cite as: | arXiv:2512.07814 [cs.SE] |
| (or arXiv:2512.07814v3 [cs.SE] for this version) | |
| https://doi.org/10.48550/arXiv.2512.07814 arXiv-issued DOI via DataCite |
Submission history
From: Hua Yang [view email]
[v1]
Mon, 8 Dec 2025 18:47:40 UTC (1,738 KB)
[v2]
Tue, 9 Dec 2025 03:23:33 UTC (1,738 KB)
[v3]
Fri, 23 Oct 2026 20:52:23 UTC (3,252 KB)
来源：arXiv cs.AI · arxiv.org