跳到正文
arXiv cs.AI· Hongsen Cao, Shanxin Yuan, Mona Jaber, Ahmed Sayed·· 4 小时前AI 评分43

EcoVGGT:基于固定键集推理的长视频高效流式前馈 3D 重建

EcoVGGT: Pinned-Key Inference for Efficient and Streamable Feed-Forward 3D Reconstruction from Long Videos

AI 导读

EcoVGGT 是一种免训练推理框架,用 31 帧均匀分布画面组成固定键集(Pinned Key Set),将长视频的逐帧成本降至常数,解决 VGGT 等前馈多视角 Transformer 跨帧注意力随帧数平方增长的问题。

正文 · 原文

View PDF HTML (experimental)

Abstract:Feed-forward multi-view transformers such as VGGT recover the cameras and geometry of a video in one pass, but the cost of their cross-frame attention grows quadratically with the frame count, and their accuracy drops on long videos. Most remedies approximate attention over all frames within a budget or select a few keys by their content. We hypothesize that the keys of a long video can be chosen without looking at its content, and our experiments on VGGT and $\pi^3$ support it: a few dozen evenly spread frames can serve as the keys of every frame. We present EcoVGGT, a training-free inference framework whose keys form a Pinned Key Set of 31 such frames. A pin pass stores their tokens in a Scene Cache, and the other frames read it in independent batches that reproduce joint inference with these keys exactly, at a constant cost per frame. An Online Stream updates the pinned frames during recording. Six middle global blocks align the frames and form a reusable VGGT module: the Alignment Score reads their attention to flag failures, and Alignment Refocusing reduces them by fine-tuning only these blocks (3.6 GPU hours). EcoVGGT ($\pi^3$) matches the success rate of $\pi^3$ on 1,000-frame ScanNet videos at 13.8 times its speed. With refocusing, EcoVGGT (VGGT) raises the success rate of VGGT from 70% to 82% at 8.5 times its speed on 500 frames. EcoVGGT reconstructs 4,000 frames on one GPU, and EcoVGGT ($\pi^3$) streams 1,000 frames at 18.6 frames per second with the lowest online absolute trajectory error.
Subjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
Cite as: arXiv:2611.18099 [cs.CV]
  (or arXiv:2611.18099v1 [cs.CV] for this version)
  https://doi.org/10.48550/arXiv.2611.18099

arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Hongsen Cao [view email]
[v1] Fri, 13 Nov 2026 12:26:54 UTC (17,659 KB)

来源:arXiv cs.AI · arxiv.org