arXiv cs.AI· Paula Chikaodinaka Anyanwu, Valdini Douglace Lemofouet, Blessing Ngozi Uzor, Ifeoluwa Dorcas Awotunde, Danielle Blanche Kapsa, Tassallah Abdullahi, Clemencia Siro, Idris Abdulmumin, Seid Muhie Yimam, Shamsuddeen Hassan Muhammad·· 3 小时前AI 评分46
低资源非洲语言红队测试：以约鲁巴语和尼日利亚皮钦语评测 LLM 安全性
Red-Teaming Large Language Models in Low-Resource African Languages: A Case Study of Yoruba and Nigerian Pidgin
AI 导读
一项研究用 400 条显式对抗提示词在 Yoruba 和 Nigerian Pidgin 中评测六个主流 LLM，发现英语中拒绝率高达 69% 的模型在 Yoruba 中拒绝率可低至 0%，主因是模型理解不了提示词而非更安全。
正文 · 原文
Abstract:As large language models (LLMs) become more widespread in Africa, their safety in low-resource languages remains largely untested since most red-teaming work focuses on English. This study evaluates six state-of-the-art LLMs using 400 explicit adversarial prompts in Yoruba and Nigerian Pidgin, categorised into four harm categories. The responses were scored by native-speaker annotators and reliability was assessed using Cohen's $\kappa$. While models that rejected up to 69% of English prompts rejected as few as 0% in Yoruba, this was not because they were safer, but because they failed to understand the prompts. Attack success rates increased for most models in the Nigerian languages, with GPT-4o Mini demonstrating the most significant cross-lingual safety discrepancy (+0.23 in Yoruba). When models did engage with adversarial prompts, many of the responses were harmful, with rates of harmful content reaching 0.63 in English and 0.50 in Nigerian Pidgin. The most difficult harm category to detect was cultural stereotypes and discrimination (ASR = 0.44 in Nigerian Pidgin). Comprehension in Yoruba was low across most models, and this directly suppressed apparent attack success without indicating actual safety. Gemini 2.0 Flash was the most consistent model across languages. Despite its explicit training in African languages, Aya 101 had the highest attack success rate in English (0.60) and one of the lowest Yoruba comprehension rates (0.28). These results demonstrate that English-centric safety alignment does not transfer to Nigerian languages, highlighting the urgent need for language-specific safety evaluations.
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.39590 [cs.CL] |
| (or arXiv:2610.39590v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.39590 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Paula Chikaodinaka Anyanwu [view email]
[v1]
Thu, 29 Oct 2026 08:30:29 UTC (2,993 KB)
来源：arXiv cs.AI · arxiv.org