arXiv cs.AI· Sheng Xu, Zhen Chen, Junhua Wang, Boyuan Huang, Ke Jia, Jiadun Zhu, Yiming Xu·· 4 小时前AI 评分40
幅值门控：一种修复 LLM 工具调用结构化输出的 FFN 干预方法与门控评测协议
Amplitude-Only FFN Intervention for Tool-Structured LLM Inference: Method, Gated Evaluation Protocol, and Cross-Model Empirical Results
AI 导读
一篇 arXiv 论文提出 Amplitude Gating（AG）方法，保持预训练权重固定，在六个 SwiGLU 位点用基于幅值/熵的乘性掩码干预 FFN 激活，以修复工具调用等结构化输出。
正文 · 原文
Abstract:We study amplitude-only rescaling of feed-forward network (FFN) activations to repair tool calls and other structured outputs, using multiplicative coordinate masks rather than additive steering vectors. Amplitude Gating (AG) keeps pretrained weights fixed and applies magnitude/entropy-based masks at six SwiGLU sites, with a Newton-Schulz windowed ranking variant. Across three Qwen checkpoints and six text/tool endpoints, effects are task dependent. Under frozen local scorers, Qwen3.5-9B non-NS configurations improve two function-call endpoints by +2.61/+2.67 percentage points (pp) and static ToolBench in/out-domain by +1.64/+2.82 pp; JSON gains are small. These gains survive comparing the weaker of two observed AG runs with the stronger baseline, supporting observed-run robustness rather than population significance. Scores measure lexical or answer/action-name agreement, not executed-tool success. Under the current experimental conditions, the six-task random-forest refit improves native score by +0.46 pp with a positive conditional paired interval; transfer to other settings requires further evaluation. The results support task-specific interventions; causal mechanisms, equal-budget advantages, repeated-run uncertainty, and serving cost require further evaluation. Code and experimental data: this https URL.
| Comments: | 46 pages, 11 figures, 30 tables. Updated six-task results and RF refit; clarified steering terminology, intervention-site analysis, and experimental scope. Code and experimental data: this https URL |
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Machine Learning (cs.LG) |
| Cite as: | arXiv:2607.11183 [cs.CL] |
| (or arXiv:2607.11183v5 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2607.11183 arXiv-issued DOI via DataCite |
Submission history
From: Zhen Chen [view email]
[v1]
Mon, 13 Jul 2026 07:28:55 UTC (2,045 KB)
[v2]
Tue, 14 Jul 2026 08:45:42 UTC (2,045 KB)
[v3]
Wed, 5 Aug 2026 06:56:38 UTC (1,708 KB)
[v4]
Thu, 15 Oct 2026 01:06:08 UTC (2,084 KB)
[v5]
Mon, 26 Oct 2026 10:41:08 UTC (2,029 KB)
来源：arXiv cs.AI · arxiv.org