arXiv cs.AI· Basel Shbita, Michael Hind, Chad DeLuca, Nathan Fulton, Shubhi Asthana, Farhan Ahmed, Anisa Halimi, Yada Zhu, Mauro Martino, David Cox, Dan Gutfreund·· 2 小时前AI 评分50
LLMON：通过后训练增强大语言模型对提示词注入攻击的防御
Post-Training Large Language Models with LLMON
AI 导读
研究团队提出 LLMON（LLM Object Notation）后训练方法，通过在上下文中显式分离指令与数据、要求基于标识符的执行，缓解大语言模型的提示词注入漏洞。团队构建了 4.6B token 的结构化训练语料，对两个 Granite-4.1 基座模型（3B 和 8B）进行监督微调加强化学习，将间接提示词注入基准上的攻击成功率从 53-100% 降至 0.02-5.6%，且不损害通用能力。
正文 · 原文
Authors:Basel Shbita, Michael Hind, Chad DeLuca, Nathan Fulton, Shubhi Asthana, Farhan Ahmed, Anisa Halimi, Yada Zhu, Mauro Martino, David Cox, Dan Gutfreund
Abstract:Large Language Models (LLMs) treat all context tokens uniformly, lacking explicit separation between executable instructions and passive data. This leads to prompt injection vulnerabilities, ambiguous instruction binding, and unreliable execution control. We present a post-training methodology using LLMON (LLM Object Notation), a framework that enables separating instructions from data and requires explicit identifier-based execution. We construct a large-scale structured training corpus of 4.6B tokens spanning general, RAG (Retrieval-Augmented Generation), and tool-calling data, augmented with distractor-infused variants in which competing instructions and irrelevant data segments are present in context but not referenced. Post-training two Granite-4.1 base models (3B and 8B) through supervised fine-tuning followed by reinforcement learning, we show that LLMON-structured supervision reduces attack success rates on indirect prompt-injection benchmarks from 53-100% to 0.02-5.6% while preserving general capabilities. We further introduce attention-based interpretability metrics that diagnose failure modes at individual transformer layers for structured inputs, revealing how our approach reshapes internal attention patterns. Our results demonstrate that explicit execution semantics can be taught through standard post-training without architectural modification.
| Subjects: | Artificial Intelligence (cs.AI); Machine Learning (cs.LG) |
| Cite as: | arXiv:2610.38120 [cs.AI] |
| (or arXiv:2610.38120v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.38120 arXiv-issued DOI via DataCite |
Submission history
From: Basel Shbita [view email]
[v1]
Wed, 28 Oct 2026 00:56:01 UTC (353 KB)
来源：arXiv cs.AI · arxiv.org