SarcasTraps:讽刺对土耳其语仇恨言论检测的影响
SarcasTraps: Effect of Sarcasm on Turkish Hate Speech Detection
AI 导读
研究者发布 SarcasTraps,一个用于分析讽刺对土耳其语仇恨言论检测影响的外部测试集,包含 479 条按仇恨言论和讽刺两维标注的数据。作者用 BERTurk 和 XLM-RoBERTa 在两个土耳其语仇恨言论数据集上训练分类器,并在零样本设置下评测 GPT-4.1 和 Llama-3.3-70B。结果显示微调模型在讽刺仇恨言论上召回率下降,零样本 LLM 召回率较高但精确率大幅下滑。
正文 · 原文
Abstract:Standard hate speech detection datasets often emphasize explicit harmful language, but directed harm can also be expressed indirectly, without clearly hateful wording. Sarcasm is one such form of indirect expression, as it can convey harmful intent through mock praise, exaggeration, humor, or apparent politeness. In this paper, we introduce SarcasTraps, a targeted external test set for analyzing the effect of sarcasm on Turkish hate speech detection. SarcasTraps contains 479 entries annotated along two dimensions: hate speech and sarcasm. We train classifiers using BERTurk and XLM-RoBERTa on two existing Turkish hate speech datasets and evaluate them on SarcasTraps, using it strictly as an external test set. We also evaluate GPT-4.1 and Llama-3.3-70B in a zero-shot setting on SarcasTraps. Results show that sarcasm substantially affects hate speech detection. Fine-tuned encoder models lose recall on sarcastic hate, while zero-shot LLMs maintain higher recall but suffer substantial precision drops on sarcastic examples. These findings show that both fine-tuned hate speech detectors and zero-shot LLMs struggle when sarcasm is involved, and SarcasTraps provides quantitative evidence of this challenge through a targeted Turkish test set. Due to the presence of sensitive and hateful language, SarcasTraps will be shared upon request for academic research purposes. Warning: This paper has examples from the dataset which contains offensive and hateful content towards groups of people.
| Comments: | 12 pages |
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Machine Learning (cs.LG) |
| Cite as: | arXiv:2610.16957 [cs.CL] |
| (or arXiv:2610.16957v2 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.16957 arXiv-issued DOI via DataCite |
Submission history
From: Dilara Keküllüoğlu [view email]
[v1]
Mon, 12 Oct 2026 15:25:27 UTC (65 KB)
[v2]
Tue, 17 Nov 2026 09:03:21 UTC (58 KB)
来源:arXiv cs.AI · arxiv.org