跳到正文
arXiv cs.AI· Feng Han, Ayako Yagahara, Kenji Hirata, Atsushi Yoshida, Kohsuke Kudo·· 56 分钟前AI 评分33

细粒度日文临床命名实体识别:BERT 与开源权重 LLM 的受控对比

Fine-Grained Japanese Clinical Named Entity Recognition: A Controlled Comparison of BERT and Open-Weight Large Language Models

AI 导读

一项研究在定义 113 个实体类别的 iCorpus 数据集上,对比了两个微调 BERT-CRF 基线与九个开源权重 LLM 在日文临床细粒度命名实体识别中的表现。

正文 · 原文

View PDF

Abstract:Background: Generative large language models (LLMs) offer an alternative to encoder-based clinical named entity recognition (NER), but their performance on fine-grained Japanese extraction tasks and sensitivity to generation settings remain uncertain. Methods: We compared two fine-tuned BERT-CRF baselines with nine open-weight LLMs on iCorpus, which defines 113 entity categories. The evaluation used 433 sentences from 36 test documents; 1,672 sentences were used for training. LLMs were evaluated using zero-shot prompting with and without JSON Schema constraints and three-seed QLoRA fine-tuning. Fine-tuned models were assessed under different output-token limits, including 900 and larger model-specific limits of 2,600 or 4,096 tokens. Performance was measured using sentence-level label-surface matching with relaxed and exact-surface F1. Results: The highest zero-shot relaxed F1 was 0.337, compared with 0.836-0.841 for the BERT baselines. At the larger output limits, all eight fine-tuned LLMs with at least 13B total parameters exceeded both BERT baselines (relaxed F1, 0.882-0.914; exact-surface F1, 0.840-0.880). Across three-seed means, increasing the output limit improved relaxed F1 by 0.052-0.085, alongside higher recall and fewer length-limited outputs. JSON Schema constraints changed relaxed F1 by -0.014 to +0.008 among seven models outside the two reasoning-oriented configurations, while medical-general model pairs showed no consistent advantage for medical specialization. Conclusions: On this historical Japanese case-report benchmark, task-specific fine-tuning combined with adequate generation capacity was associated with stronger LLM extraction performance. Output-token limits materially affected comparisons with BERT, whereas enforcing output structure alone did not overcome low zero-shot accuracy.
Comments: 33 pages, 3 figures, 5 tables; includes appendices with supplementary tables and one supplementary figure
Subjects: Artificial Intelligence (cs.AI); Computation and Language (cs.CL)
ACM classes: I.2.7; J.3
Cite as: arXiv:2611.22219 [cs.AI]
  (or arXiv:2611.22219v1 [cs.AI] for this version)
  https://doi.org/10.48550/arXiv.2611.22219

arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Feng Han PhD. [view email]
[v1] Tue, 17 Nov 2026 05:01:24 UTC (1,574 KB)

来源:arXiv cs.AI · arxiv.org