关系组合作为平行输运：逐实体标架对注意力机制施加了什么约束
Relational Composition as Parallel Transport: What Per-Entity Frames Force in Attention
AI 导读
一篇 arXiv 论文（arXiv:2610.34945）把逐实体坐标系的自由视为一种对称性，推导它在注意力中强制出现的结构：不变打分只依赖范数与输运内积，等变消息落在 x_i 与 T_ij x_j 张成的空间中，标准注意力则是共享标架的平坦分支。
正文 · 原文
Abstract:A dot product means something only when both vectors share a coordinate system. Content words have one; a fresh entity does not: rotating its coordinates alone changes nothing. We take this per-entity frame freedom as a symmetry and ask what it forces in attention. For orthogonal transports, invariant scores depend only on norms and the transported inner product $\langle x_i,T_{ij}x_j\rangle$; equivariant messages lie in the span of $x_i$ and $T_{ij}x_j$. Endpoint channel count bounds transport rank, so with one vector per entity relations must be supplied; standard attention is the flat sector of a shared frame. Representation theory attaches an integer to each mechanism; the readout decides which. With orthogonal transitions, a convex-cell readout composes a finite group exactly only at or above its smallest faithful real representation dimension. At this minimum, exactness forces a faithful representation. An even readout can decode a cover at smaller width, and the gap is unbounded. Below its threshold, the part of the group a mechanism cannot see sets an accuracy ceiling computable before training. 17/17 fitting seeds at minimal width passed tests through length 1000, versus 7/18 at twice the width; 12 models carry correctness certificates to explicit lengths, from the weights alone. On $C_3\times A_4$ (faithful width 5), descent at width 4 learned the 72-state double cover under sign-blind attention (6/48 seeds; monotone 0/48); on a frozen cover the cross-entropy-optimal affine head is uniform, a quadratic head exact. On parsed CLUTRR, transported composition beats a matched transformer by 41 points at 4-10 hops, but forcing isometries costs 65 points; on StepGame's commutative algebra, isometries help. Static embeddings gave each word one vector; contextual models gave each occurrence its own; per-entity frames make each relation an operator, the unit in which relations compose.
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Formal Languages and Automata Theory (cs.FL) |
| Cite as: | arXiv:2610.34945 [cs.LG] |
| (or arXiv:2610.34945v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2610.34945 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Aakash Sarkar [view email]
[v1]
Mon, 26 Oct 2026 07:11:23 UTC (230 KB)
来源：arXiv cs.AI · arxiv.org