arXiv cs.AI· Omar Farouk Zouak, Selma Mani, Larbi Said Chikh, Raid Abderrezak Ouahioune·· 2 小时前AI 评分39
查询的记忆程度如何决定经典阿拉伯语诗歌的词汇检索与稠密检索效果
What the Query Remembers: Query Abstraction Governs Lexical and Dense Retrieval of Classical Arabic Poetry
AI 导读
一项针对经典阿拉伯语诗歌检索的研究表明，检索器孰优孰劣取决于查询中保留了多少诗歌原文。在“话到嘴边”类查询上，BM25 以 0.746 对 0.717 的 Recall@5 领先包括 8B 模型在内的全部九个稠密编码器；在抽象查询上则跌至十名中的第八（0.392 对 0.748）。同一查询改写措辞即可令 BM25 提升 38 分，而只报告单一评测协议或查询类型的基准可能给出严重偏差的结论。
正文 · 原文
Abstract:How well a poetry archive can be searched depends on what the user still remembers, not only on what the model understands. We show this for Classical Arabic poetry, benchmarking nine dense encoders and BM25 over two corpora, three indexing modes and two evaluation protocols. Which retriever wins is governed by how much of a poem's wording the query still carries. On tip-of-the-tongue queries BM25 leads every dense encoder, including an 8B state-of-the-art model (0.746 vs. 0.717 Recall@5); on abstract queries it falls to eighth out of ten (0.392 vs. 0.748). Editing the wording of the same queries shows that this ranking largely reflects query composition: adding one remembered half-verse raises BM25 by 38 points, rewriting the quoted words by hand leaves BM25 level with the 8B encoder, and BM25's sensitivity to remembered wording is significantly steeper than that of the strongest dense encoder. The same axis controls explanation augmentation, whose effect swings from +37 to -39 points. The mechanism splits by system type: lexical containment predicts BM25's behaviour, while embedding proximity predicts dense encoders. Finally, the field's two standard protocols, known-item retrieval and thematic relevance, dissociate on the same systems, a result reproduced by two blind human annotators and by two open-weights judges, one of them under a rubric with no literal-match criterion. A benchmark that reports one protocol, or one query register, can give a materially wrong picture. Code and data are available at this https URL.
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Information Retrieval (cs.IR) |
| Cite as: | arXiv:2610.31821 [cs.CL] |
| (or arXiv:2610.31821v1 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2610.31821 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Omar Farouk Zouak [view email]
[v1]
Thu, 22 Oct 2026 18:59:15 UTC (625 KB)
来源：arXiv cs.AI · arxiv.org