arXiv cs.AI· Maria Elkj{\ae}r Montgomery, Christian Igel, Mikkel Odgaard, Martin Sillesen, Mads Nielsen·· 4 小时前AI 评分39
Transformer 在电子健康记录中应如何编码数值？
How Should Transformers Encode Numeric Values in Electronic Health Records?
AI 导读
一项 ICML 2026 论文系统比较了 Transformer 处理电子健康记录（EHR）数值时的离散、连续和混合编码策略，采用嵌入真实 EHR 数据的合成算术任务和真实临床预测任务评估。
正文 · 原文
Abstract:How do we encode numeric values in transformer-based sequence processing, particularly in electronic health record (EHR) data? We systematically compare discrete, continuous, and hybrid value encoding strategies using synthetic arithmetic tasks embedded within real-world EHR data, as well as real-world clinical prediction tasks. Our study reveals trade-offs between numeric precision, optimisation stability, and architectural flexibility. We find that approaches that explicitly model value-concept interactions perform best on precision-sensitive arithmetic tasks when architectural constraints permit. Hybrid token-based approaches that retain numeric values but apply binning prior to projection provide a more robust and broadly applicable alternative, with the optimal number of bins following a simple empirically derived power-law in dataset size. Across tasks, models consistently exhibit reliable "good enough" numeric computation rather than exact arithmetic, while clinical gains from incorporating laboratory values are task-dependent. This suggests that robustness and deployability often outweigh maximal numeric precision in practice, motivating hybrid token-based approaches as a practical default.
| Comments: | 16 pages, 15 figures, 3 tables, accepted to ICML 2026 |
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2607.01391 [cs.LG] |
| (or arXiv:2607.01391v2 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2607.01391 arXiv-issued DOI via DataCite |
|
| Journal reference: | Proceedings of the 43rd International Conference on Machine Learning (2026) |
Submission history
From: Maria Elkjær Montgomery [view email]
[v1]
Wed, 1 Jul 2026 18:49:41 UTC (6,177 KB)
[v2]
Fri, 23 Oct 2026 13:14:27 UTC (6,177 KB)
来源：arXiv cs.AI · arxiv.org