arXiv cs.AI· Isaac Saxton-Knight, Gabriel Saxton-Knight, Diego Oliveira, Tidor-Vlad Pricope, Aston Garratt, Jocelyn D'Arcy·· 4 小时前AI 评分34
CulturalCrosswords：多语言文化谜题答案约束与填字网格补全基准
CulturalCrosswords: A Multilingual Benchmark of Answer Constraints and Grid Completion
AI 导读
研究者构建了 CulturalCrosswords 基准，用 42 种语言的众包谜题组成 53 个填字游戏，评测 11 个 LLM 配置在五种提示形式下的表现。在有备选答案的 962 条谜题上，加入长度约束使与参考答案的一致率提升 22.8 个百分点，而给备选答案计分时仅提升 8.4 个百分点。填字输入使交叉点答案兼容率提高 9.7 个百分点，要求补全整个网格则揭示出模型间的显著差异。
正文 · 原文
Abstract:When several answers fit a riddle, exact-match scoring can penalise a valid response. CulturalCrosswords tests how answer constraints affect large language models (LLMs) solving cultural riddles, whose solutions draw on knowledge specific to a community. We constructed 53 crosswords from crowdsourced riddles in 42 languages and evaluated 11 LLM configurations on five versions of each clue set: plain batches, batches with lengths, batches with lengths and partial-answer hints, and crosswords requesting an answer list or a completed grid. On 962 clues with alternative answers, lengths improve agreement with the reference by 22.8 percentage points, compared with 8.4 points when accepted alternatives also receive credit. Part of the gain therefore appears to come from selecting a required answer form. Crossword input increases the share of all crossings with compatible answers by 9.7 points; requiring a completed grid reveals large differences between models. The benchmark distinguishes agreement with a reference from the ability to produce answers that fit together and occupy the correct cells.
| Comments: | 23 pages, 7 figures, including appendices |
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2611.10044 [cs.CL] |
| (or arXiv:2611.10044v2 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2611.10044 arXiv-issued DOI via DataCite |
Submission history
From: Tidor-Vlad Pricope [view email]
[v1]
Thu, 5 Nov 2026 21:45:08 UTC (197 KB)
[v2]
Tue, 10 Nov 2026 03:31:27 UTC (197 KB)
来源：arXiv cs.AI · arxiv.org