LLM 多实例处理性能退化研究：实例数量比上下文长度影响更大

View PDF HTML (experimental)

Abstract:Users often rely on Large Language Models (LLMs) for processing multiple documents or performing analysis over a number of instances. For example, analysing the overall sentiment of a number of movie reviews requires an LLM to process the sentiment of each review individually in order to provide a final aggregated answer. While LLM performance on such individual tasks is generally high, there has been little research on how LLMs perform when dealing with multi-instance inputs. In this paper, we perform a comprehensive evaluation of the multi-instance processing (MIP) ability of LLMs for tasks in which they excel individually. The results show that all LLMs follow a pattern of slight performance degradation for small numbers of instances (approximately 20-100), followed by a performance collapse on larger instance counts. Crucially, our analysis shows that while context length is associated with this degradation, the number of instances has a stronger effect on the final results. This finding suggests that when optimising LLM performance for MIP, attention should be paid to both context length and, in particular, instance count.

Submission history

From: Jingxuan Chen [view email]
[v1] Mon, 23 Mar 2026 22:13:15 UTC (638 KB)
[v2] Mon, 20 Apr 2026 20:06:22 UTC (641 KB)
[v3] Wed, 11 Nov 2026 16:33:09 UTC (571 KB)