跳到正文
arXiv cs.AI· Davide Liga, Liuwen Yu, Matteo Magnini·· 56 分钟前AI 评分47

解析语言模型的道德回路与道德特征:来自 14 个模型的证据

Disambiguating Moral Circuits and Moral Features in Language Models: Moral content, normative judgment or cue words? Evidence from 14 models

AI 导读

一项可解释性研究用 227 组经五名标注者验证的最小对短语,测试 Llama、Gemma、Qwen 共 14 个开源权重模型(1B 到 14B)。

正文 · 原文

View PDF HTML (experimental)

Abstract:Interpretability methods can locate the attention heads and sparse autoencoder (SAE) features that change the answer a language model gives to a moral prompt, e.g., whether "Stealing medicine for a dying child is morally" continues with "justified" or "wrong". Such a component could respond to moral content, to the judgment of conduct by any norm, or to the word "morally" in the prompt. We built 227 minimal pairs of short phrases, validated by five annotators, whose two versions differ in a moral standard, in another kind of norm (prudence, etiquette, aesthetics or evidence) or in a descriptive quality, and showed them to 14 open-weight models (Llama, Gemma and Qwen, 1B to 14B, base and instruction-tuned), with and without the word "morally". In every model, the heads that change the moral answer belong to an evaluation circuit the models use for every kind of normative judgment we tested: these heads carry the same share of the effect for moral as for other normative pairs (pooled ratio 1.00, 97.5% confidence interval [0.92, 1.07], with the word; 0.90, [0.84, 0.96], without it), and the models need them as much for both. The models need these heads also when nothing in the prompt asks for a judgment, and on 400 scenarios of the ETHICS benchmark, where removing them takes away 37 to 39% of the difference between acceptable and unacceptable scenarios, against 1 to 3% for ten random heads. The SAE features we located respond mostly to the word "morally" rather than to what the phrase is about. The models represent the category of a pair (the standard the annotators assigned to it): it can be read from their hidden states, and, in the eight models we tested, removing the direction that carries it weakens the moral verdict by a few percent, and more than the other verdicts in seven of them. We release the pairs, the ratings, the code, the trained SAE dictionaries and the result files.
Subjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Machine Learning (cs.LG)
Cite as: arXiv:2611.21705 [cs.CL]
  (or arXiv:2611.21705v1 [cs.CL] for this version)
  https://doi.org/10.48550/arXiv.2611.21705

arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Davide Liga [view email]
[v1] Mon, 16 Nov 2026 16:12:45 UTC (394 KB)

来源:arXiv cs.AI · arxiv.org