跳到正文
arXiv cs.AI· Yiyang Zhu, Guangyuan Liu, Jingdi Lei, Li Wei, Chau Yuen·· 1 小时前AI 评分38

从点到轨迹:CounterPoint 重新思考物理事件的多模态对齐

From Points to Trajectories: Rethinking Multimodal Alignment for Physical Events

AI 导读

arXiv 论文提出 CounterPoint,一个轻量即插即用模块,解决多模态对齐在物理事件上的失效问题。现有架构将时间序列压缩为单一池化嵌入,导致在“孪生事件”(相同任务条件下的不同物理执行)上无法区分,且序列越长 pooled 表示越收敛于静止录制条件。CounterPoint 通过剥离各模态时间均值、逐帧评分跨模态一致性,在四个 SOTA 多模态对齐器上集成后均优于现有时序对齐方法。

正文 · 原文

View PDF HTML (experimental)

Abstract:Advancing physical artificial intelligence (Physical AI) requires moving beyond static object recognition to ground how real-world mechanics (e.g., impact, friction, and force transfer) unfold over time. While multimodal aligners seek to capture these interactions by matching multi-sensor streams, prevailing architectures compress each temporal sequence into a single pooled embedding. We show that this paradigm fails on twin events: distinct physical executions recorded under identical task conditions. Collapsing the temporal dimension averages out transient cross-modal dynamics, effectively reducing physical event alignment to static scene recognition. Formally, we prove that as sequence length increases, any temporally pooled representation converges to the stationary recording condition. Consequently, as windows grow, no choice of loss function or embedding geometry can disambiguate twin events from pooled features alone. Instead, we show that physical events are uniquely identified by the event gap, namely the instantaneous, synchronous co-fluctuations across modalities that are erased by pooling and accumulate only along unpooled trajectories. To recover this signal, we introduce CounterPoint, a lightweight plug-in module that strips out each modality's temporal mean to score frame-level cross-modal agreement. Integrated across four state-of-the-art (SOTA) multimodal aligners, CounterPoint shifts representations from coarse condition matching to fine-grained physical event alignment, consistently outperforming existing temporal alignment methods.
Subjects: Artificial Intelligence (cs.AI); Signal Processing (eess.SP)
Cite as: arXiv:2611.24249 [cs.AI]
  (or arXiv:2611.24249v1 [cs.AI] for this version)
  https://doi.org/10.48550/arXiv.2611.24249

arXiv-issued DOI via DataCite (pending registration)

Submission history

From: Yiyang Zhu [view email]
[v1] Wed, 18 Nov 2026 11:26:08 UTC (1,238 KB)

来源:arXiv cs.AI · arxiv.org