arXiv cs.AI· Frederic Andriantseheno Rakoto (Mnemosyne), Yannis Bendi-Ouis (Mnemosyne), Xavier Hinaut (Mnemosyne)·· 4 小时前AI 评分32
论文评测:本地部署大语言模型在科研论文检索中的成功率与推理延迟(Mac Studio vs NVIDIA A100)
Evaluating Large Language Models for Scientific Paper Search: Retrieval Success and Inference Latency on Mac Studio and NVIDIA A100
AI 导读
一项 arXiv 研究评估开源权重模型在本地部署下辅助科研论文检索的可行性。研究在 Mac Studio 和双 NVIDIA A100 服务器上测量输入上下文从 512 到 128k token 的推理延迟,并测试 8 个模型对来自 7 个科研仓库的 70 篇论文的检索效果。
正文 · 原文
Abstract:Scientific literature search can benefit from locally deployed LLM assistants, which reduce reliance on hosted model providers while giving research teams greater control over their data and infrastructure. This work investigates whether open-weight models can support web-assisted paper retrieval under practical computing constraints. We first characterize inference latency as the input context grows from 512 to 128k tokens on a Mac Studio and a server equipped with two NVIDIA A100 GPUs. We then evaluate eight models on the retrieval of 70 papers from seven scientific repositories, using queries that represent three levels of domain knowledge. The results show that retrieval success depends strongly on the selected model and the specificity of the query. And that A100 GPUs are generally faster than the Mac Studio, but its unified memory allows larger models that could not be loaded in the tested GPU configuration.
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2611.17929 [cs.AI] |
| (or arXiv:2611.17929v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2611.17929 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Yannis Bendi-Ouis [view email] [via CCSD proxy]
[v1]
Fri, 13 Nov 2026 09:08:17 UTC (82 KB)
来源:arXiv cs.AI · arxiv.org