30 年时间上下文用于初级保健 EHR 结直肠癌预测模型

View PDF HTML (experimental)

Abstract:EHR foundation models read medical events within a fixed context window, and long context has so far been measured only in tokens. How far back in time a model must look to capture the earliest predictive signal is unknown. We introduce time-context, the calendar lookback window before the prediction point, and provide, to our knowledge, the first systematic evaluation of decade-scale time-context for an EHR foundation model. We pretrained a decoder-only transformer (34.6M parameters) on 1.4B coded events from 3.6M primary-care patients in North West London and fine-tuned it to predict incident colorectal cancer under a 12-month pre-diagnostic buffer. A frozen cohort of 1,008 cases and 9,582 matched controls, all registered for at least 30 years, was evaluated at 15 nested lookback windows from 2 to 30 years, so that every truncation is compared within the same patients. Discrimination rose from an AUROC of 0.551 at 2 years to 0.590 at 18 years, then plateaued, with about 80% of the gain reached within 10 years; an XGBoost baseline converged to the same level (peak AUROC 0.589). A simulation further suggests that rotary position embeddings (RoPE) over elapsed time alias beyond about 20 years at the default base frequency, motivating a retuned $\theta$. Time-context is a measurable design axis for clinical prediction models, distinct from token-context, and should be reported.

Comments:	Accepted at Machine Learning for Health (ML4H) 2026, Findings Track. 10 pages, 6 figures, 3 tables
Subjects:	Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Quantitative Methods (q-bio.QM)
ACM classes:	I.2.6; J.3
Cite as:	arXiv:2611.07265 [cs.LG]
	(or arXiv:2611.07265v1 [cs.LG] for this version)
	https://doi.org/10.48550/arXiv.2611.07265 arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Achille Raviez [view email]
[v1] Thu, 5 Nov 2026 00:34:47 UTC (119 KB)