跳到正文
arXiv cs.AI· Muhammed Izzet Sezer, Sevval Belkis Dikkaya·· 4 小时前AI 评分36

RAWATTA:无需训练的多视角 3D Transformer 注意力加速方法

Training-Free Acceleration of Multi-View 3D Transformers via Regime-Aware Attention Approximation

AI 导读

研究者提出 RAWATTA,一种无需训练的多视角 3D Transformer 全局注意力加速方法。它发现各注意力层分为场景平均、跨图像匹配和首图锚定三种角色,仅用 8-bit 精度运行注意力,并对必需的层保留 16-bit 精度。

正文 · 原文

View PDF HTML (experimental)

Abstract:Multi-view 3D transformers such as VGGT reconstruct a scene from a set of unposed images in a single forward pass, predicting camera poses, depth and 3D points, but the cost of their global attention grows quadratically with the num- ber of images. Accelerators borrowed from large language models assume that each token attends to only a few oth- ers; on these models they lose accuracy or gain little speed. We study the global attention of three multi-view transform- ers on seven datasets and find that every layer plays one of three roles: it averages over the whole scene, matches points across many images, or anchors on the first image, whose coordinates the outputs use. Building on this anal- ysis, we propose Regime-AWare ATTention Approximation (RAWATTA), a training-free method that runs attention in 8- bit precision and keeps 16-bit precision only in the layers where a worst-case calibration check requires it. Its fast set- ting, RAWATTA-Fast, also approximates attention in a few averaging and anchoring layers, while matching layers and the first image keep all their keys. On a 12 GB GPU at 50 im- ages, RAWATTA runs VGGT faster than the released code, mostly through our CUDA engine, and loses at most about one point of pose accuracy (AUC@3) on any dataset; RAWATTA-Fast is slightly faster and less accurate. Both fit 200 images, where the released code runs out of memory at 100.
Subjects: Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
Cite as: arXiv:2611.17629 [cs.CV]
  (or arXiv:2611.17629v1 [cs.CV] for this version)
  https://doi.org/10.48550/arXiv.2611.17629

arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Muhammed Sezer [view email]
[v1] Fri, 13 Nov 2026 00:59:07 UTC (1,382 KB)

来源:arXiv cs.AI · arxiv.org