Retina 论文提出 lakehouse 实时事务性数据变更框架并开源
Transactional Data Change for Real-Time Analytics in Lakehouses
AI 导读
arXiv 论文提出 Retina，一个面向 lakehouse 的数据同步框架，实现实时、低成本、高性能且完全事务性的数据变更，已开源。评测显示其达到 10 毫秒级数据新鲜度和超过 3.2 million row/s 的线性可扩展吞吐，显著优于现有 lakehouse 系统和 lake 兼容 HTAP 数据库。论文已被 VLDB 2026 接收，将发表于 PVLDB Vol.19 No.13。
正文 · 原文
Abstract:Real-time data analytics has become a key demand for lakehouses. It requires propagating data changes from the data source to the lakehouse in real time, while ensuring ACID guarantees and minimizing resource consumption and impact on analytical query performance. This is highly challenging for a data system according to the RUM conjecture, especially given the read-optimized immutable file storage layer in lakehouses. Existing lakehouses offer only git-style catalog version control on a single table, and they trade data freshness, query performance, and/or resource efficiency to achieve a barely acceptable data-change throughput. In this paper, we propose a novel data synchronization framework, Retina, that enables real-time, cost-efficient, query-performant, and fully transactional data changes in lakehouses. We implement and open-source Retina. Evaluations show that Retina delivers 10-millisecond-level data freshness and over 3.2 million row/s linear-scalable data-change throughput with low resource consumption and query-performance overhead, significantly outperforming state-of-the-art lakehouse systems and lake-compatible HTAP databases.
| Comments: | Accepted by VLDB 2026 and to be published on PVLDB Vol.19 No.13, 14 pages, 16 figures |
| Subjects: | Databases (cs.DB); Artificial Intelligence (cs.AI); Distributed, Parallel, and Cluster Computing (cs.DC); Performance (cs.PF) |
| MSC classes: | 68T09 (Primary) 68P20, 68P05 (Secondary) |
| Cite as: | arXiv:2611.12169 [cs.DB] |
| (or arXiv:2611.12169v2 [cs.DB] for this version) | |
| https://doi.org/10.48550/arXiv.2611.12169 arXiv-issued DOI via DataCite |
|
| Related DOI: | https://doi.org/10.14778/3849398.3849411
DOI(s) linking to related resources |
Submission history
From: Haoqiong Bian [view email]
[v1]
Mon, 9 Nov 2026 10:23:17 UTC (887 KB)
[v2]
Wed, 11 Nov 2026 03:53:55 UTC (572 KB)
来源：arXiv cs.AI · arxiv.org