SALT:用跨度级监督改进跨语言 token 表示的后训练方法
Improving Cross-Lingual Token Representations by Adding a Pinch of SALT
AI 导读
SALT 是一种轻量级后训练方法,通过向现有句子编码器注入跨度级监督来改进跨语言 token 表示。在五个多语言 token 级基准中,SALT 在四个上取得最佳整体结果,优于其他微调策略和竞争编码器,同时提升跨语言检索与分类等句子级任务表现。
正文 · 原文
Abstract:Cross-lingual sentence encoders enable scalable transfer across hundreds of languages, powering applications such as translation mining and zero-shot learning in low-resource settings. Although trained for sentence-level alignment, they are increasingly also applied to token-level tasks such as hallucination detection and sequence tagging, exposing a mismatch between training and usage. We propose SALT, a lightweight post-training method that improves token representations by injecting span-level supervision into existing sentence encoders. Across five multilingual token-level benchmarks, SALT achieves the best overall results on four of them, outperforming alternative fine-tuning strategies and competitive encoders. It also improves sentence-level performance on cross-lingual retrieval and classification tasks. These results demonstrate that span-level supervision is an effective signal for improving both token and sentence representations.
| Subjects: | Computation and Language (cs.CL); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2609.09953 [cs.CL] |
| (or arXiv:2609.09953v2 [cs.CL] for this version) | |
| https://doi.org/10.48550/arXiv.2609.09953 arXiv-issued DOI via DataCite |
Submission history
From: Guillem Ramírez [view email]
[v1]
Wed, 9 Sep 2026 09:43:41 UTC (487 KB)
[v2]
Wed, 18 Nov 2026 17:40:23 UTC (2,141 KB)
来源:arXiv cs.AI · arxiv.org