跳到正文
arXiv cs.AI· Guillem Ram\'irez·· 2 小时前AI 评分38

SALT:用跨度级监督改进跨语言 token 表示的后训练方法

Improving Cross-Lingual Token Representations by Adding a Pinch of SALT

AI 导读

SALT 是一种轻量级后训练方法,通过向现有句子编码器注入跨度级监督来改进跨语言 token 表示。在五个多语言 token 级基准中,SALT 在四个上取得最佳整体结果,优于其他微调策略和竞争编码器,同时提升跨语言检索与分类等句子级任务表现。

正文 · 原文

View PDF HTML (experimental)

Abstract:Cross-lingual sentence encoders enable scalable transfer across hundreds of languages, powering applications such as translation mining and zero-shot learning in low-resource settings. Although trained for sentence-level alignment, they are increasingly also applied to token-level tasks such as hallucination detection and sequence tagging, exposing a mismatch between training and usage. We propose SALT, a lightweight post-training method that improves token representations by injecting span-level supervision into existing sentence encoders. Across five multilingual token-level benchmarks, SALT achieves the best overall results on four of them, outperforming alternative fine-tuning strategies and competitive encoders. It also improves sentence-level performance on cross-lingual retrieval and classification tasks. These results demonstrate that span-level supervision is an effective signal for improving both token and sentence representations.
Subjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
Cite as: arXiv:2609.09953 [cs.CL]
  (or arXiv:2609.09953v2 [cs.CL] for this version)
  https://doi.org/10.48550/arXiv.2609.09953

arXiv-issued DOI via DataCite

Submission history

From: Guillem Ramírez [view email]
[v1] Wed, 9 Sep 2026 09:43:41 UTC (487 KB)
[v2] Wed, 18 Nov 2026 17:40:23 UTC (2,141 KB)

来源:arXiv cs.AI · arxiv.org