GRAP：基于性别评分维度的探测框架，分析 LLM 性别偏见
GRAP: Gender Rubric-Aware Probing for Analyzing Gender Bias in LLMs
AI 导读
一篇被 ECML-PKDD 2026 AI 偏见与公平研讨会接收的论文提出 GRAP（Gender Rubric-Aware Probing）框架，用于分析 LLM 开放式回复中的性别偏见。该框架对回复按 10 个评分维度打分，用轻量级 logistic 探测器找出最具区分力的偏见维度，且不依赖特定任务。相比依赖代词消解等固定格式任务的现有方法，GRAP 解决了组级分数对比忽略维度方差的问题。
正文 · 原文
Abstract:As LLMs increasingly mediate everyday interactions, subtle gender biases in open-ended responses pose growing societal risks. Existing probing methods have two key limitations: they rely on fixed-format tasks, such as pronoun resolution or occupation association, that alignment-tuned models can easily handle; and they compare raw group-level rubric scores, ignoring per-dimension variance and obscuring which bias dimensions reliably distinguish gender groups. We introduce GRAP (Gender Rubric-Aware Probing), a framework that uses a lightweight logistic probe to identify the most discriminative bias dimensions from responses scored across 10 rubric dimensions, operating task-agnostically on open-ended instruction following.
| Comments: | 3 pages. Accepted at the ECML-PKDD 2026 Workshop on Bias and Fairness in AI |
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2611.13024 [cs.AI] |
| (or arXiv:2611.13024v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2611.13024 arXiv-issued DOI via DataCite |
Submission history
From: Seona Seo [view email]
[v1]
Mon, 9 Nov 2026 08:09:28 UTC (3,364 KB)
来源：arXiv cs.AI · arxiv.org