延迟标签与分布偏移下数据流评估的时序代数框架
A Temporal Algebraic Framework for Data Stream Evaluation Under Delayed Labels and Distribution Shifts
AI 导读
Ronaldo Prati 在 arXiv 论文中提出一种面向数据流的时序代数评估框架,将混淆矩阵的 micro/macro 度量扩展到流式场景,用指数衰减或滑动窗口加权,每步更新代价 O(m^2)。
正文 · 原文
Abstract:In data streams, labels often arrive late or only for some instances, class prevalence drifts, and rare classes vanish from short windows. Standard evaluation then confuses a degraded decision boundary with a change in prevalence, and is biased when labels are delayed or selected by the model. We extend an algebraic framework that obtains pooled (micro) and per-class (macro) measures from one confusion matrix to streams, weighting the counts in time by exponential fading or a sliding window; the update costs O(m^2) per step, whatever the stream length, and each pooled measure stays a weighted combination of the per-class ones. Delayed and selective feedback enter as an observation mask: until its label arrives, an instance is counted with the model's calibrated class probabilities, and the bias of this interim estimate is driven by their calibration error. Audited conventions report no value for a class without verified instances. We separate prior shift from concept drift by monitoring prior-free coordinates (recall and specificity) and attribute a change in a measure to the two sources with a first-order Jacobian decomposition. On synthetic streams and four real datasets, a detector on these coordinates with a memory per class raises no excess false alarms under label shift, unlike the Jacobian alarm and error-rate detectors. The correction for delayed labels recovers recall in some real settings and is only partial for precision, specificity and F1. Twenty-five statements are checked in Lean 4, and the code is in the aggmetrics package.
| Comments: | 53 pages, 25 tables, 2 figures. Code: this https URL (experiments), this https URL (Lean 4 formalization), this https URL (aggmetrics >= 0.4.0, subpackage this http URL) |
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Databases (cs.DB) |
| MSC classes: | 68T05, 62H30, 62L10, 68V20 |
| ACM classes: | I.2.6; H.2.8; G.3 |
| Cite as: | arXiv:2611.21914 [cs.LG] |
| (or arXiv:2611.21914v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2611.21914 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Ronaldo Prati [view email]
[v1]
Mon, 16 Nov 2026 21:28:59 UTC (301 KB)
来源:arXiv cs.AI · arxiv.org