arXiv cs.AI· Qiang Chen, Xiao Wang, Qingquan Yang, Hao Si, Zikang Yan, Guosheng Xu, Jin Tang·· 2 小时前AI 评分35
FusionCLIP:面向聚变等离子体诊断的多模态对比预训练框架
FusionCLIP: Multimodal Contrastive Pretraining for Fusion Plasma Diagnostics
AI 导读
研究者提出多模态对比预训练框架 FusionCLIP,用选择性视频编码器与基于查询的时间序列编码器对齐托卡马克可见光视频与诊断信号表征,并构建含 1307 次 EAST 放电的 EAST-MMD 数据集。模型在破裂预测、ELM 分类和 H98 回归三项微调任务上取得 89.07% 的 Joint score,较最佳基线分别绝对提升 2.41% 和 2.90%。
正文 · 原文
Abstract:Visible-light videos and multivariate diagnostic signals provide complementary observations of tokamak discharges. However, existing fusion pretraining methods primarily learn representations by reconstructing or predicting observations, leaving the correspondence between these modalities underexplored. Representation learning is further challenged by spatially localized plasma regions in video frames and substantial differences in diagnostic sampling rates. We propose FusionCLIP, a multimodal contrastive pretraining framework. A selective video encoder retains informative visual tokens, while a query-based time-series encoder integrates asynchronous diagnostic signals using their observation timestamps. Contrastive learning aligns representations of the two modalities. We construct the EAST Multimodal Diagnostics (EAST-MMD) dataset, comprising 1307 EAST discharges, and evaluate the model through independent fine-tuning on three tasks: disruption prediction, edge-localized mode (ELM) classification, and H98 regression. FusionCLIP achieves a Joint score of 89.07%, with absolute gains of 2.41% and 2.90% over the best competing baseline, and the in-domain contrastive pretraining baseline CLIP-Base, respectively. Ablations and attention visualizations further support the contributions of its key components.
| Subjects: | Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2611.09493 [cs.CV] |
| (or arXiv:2611.09493v2 [cs.CV] for this version) | |
| https://doi.org/10.48550/arXiv.2611.09493 arXiv-issued DOI via DataCite |
Submission history
From: Qiang Chen [view email]
[v1]
Fri, 6 Nov 2026 14:14:41 UTC (7,563 KB)
[v2]
Wed, 18 Nov 2026 11:20:14 UTC (8,047 KB)
来源:arXiv cs.AI · arxiv.org