arXiv cs.AI· Gerard Donahue, Guven Gergerli, Ayush Gupta, Reza Ghoddoosian, Enna Sachdeva, Faizan Siddiqui·· 3 小时前AI 评分35
ARQ-VAE：面向内存高效时序动作分割的自适应残差量化方法
Adaptive Residual Quantization for Memory-Efficient Temporal Action Segmentation
AI 导读
研究者提出 ARQ-VAE，一种免标注的紧凑特征表示，通过逐帧自适应截断残差量化链来压缩时序动作分割（TAS）的视频特征存储。在 Breakfast 基准上，训练集存储从 28 GB 的 I3D 特征库降至 10 MB，配合 ASFormer 在有监督设置下取得 71.5 Edit 和 51.2 F1@50。同一表示在增量式与无监督 TAS 中同样有效。
正文 · 原文
Abstract:Temporal action segmentation (TAS) is commonly trained on large banks of pre-extracted frame features, but storing and replaying those features becomes a practical bottleneck for long-form video, especially in incremental TAS where past-task data must remain available to mitigate catastrophic forgetting. Recent TAS replay and condensation methods reduce this burden, but they rely on dense frame-wise annotations and allocate storage through fixed segment-level structures, limiting their flexibility for long procedural videos whose frame complexity varies substantially over time. We introduce Adaptive Residual-Quantized Variational AutoEncoder (ARQ-VAE), a compact and label-free memory representation for TAS features that learns a discrete residual quantization of frame-level video features and then adaptively truncates the residual chain on a per-frame basis, retaining only the residual prefix that best reconstructs each frame feature. This yields a practical compact feature memory that does not depend on frame-wise labels and can therefore support multiple TAS settings with the same stored representation, while a lightweight fine-tuning stage further improves reconstruction quality for the truncated codes that are actually retained. Across standard TAS benchmarks, ARQ-VAE achieves a substantially improved storage-performance trade-off over prior replay and condensation baselines. On Breakfast, our final representation reduces training-set storage from 28 GB for the original I3D feature bank to 10 MB while retaining strong downstream segmentation performance; with ASFormer, it achieves 71.5 Edit and 51.2 F1@50 in the supervised setting. We further show that the same label-free memory remains effective for incremental and unsupervised TAS. We position ARQ-VAE as a practical memory representation for TAS and a promising direction for broader dataset condensation applications.
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2611.13262 [cs.CV] |
| (or arXiv:2611.13262v1 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2611.13262 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Faizan Siddiqui [view email]
[v1]
Mon, 9 Nov 2026 21:31:31 UTC (882 KB)
来源：arXiv cs.AI · arxiv.org