arXiv cs.AI· Yuxiang Zeng, Kun Xie, Yang Wang, Jigang Wen, Xiaocan Li, Guangxing Zhang, Gaogang Xie, Jiannong Cao·· 5 小时前AI 评分28
DTA-GT：面向神经网络表示学习的方向与拓扑感知 Graph Transformer
DTA-GT: Direction- and Topology-Aware Graph Transformer for Neural Network Representation Learning
AI 导读
DTA-GT 是一种方向与拓扑感知的 Graph Transformer，用于神经架构搜索中的精度与延迟预测，已被 NeurIPS 2026 接收。它提出"编码器全局方向一致性"设计原则，整合 Magnetic Laplacian 谱编码、方向对偶结构注意力和方向敏感的 MoE-FFN。
正文 · 原文
Abstract:Architecture attribute predictors reduce the cost of neural architecture search by estimating accuracy or latency from a small set of evaluated candidates. Neural architectures are operation-labeled directed acyclic graphs (DAGs), where edge directions determine computational roles and execution dependencies. Existing graph Transformer predictors incorporate topological and directional information in different ways, but primarily focus on specific mechanisms rather than coordinated encoder-wide modeling of these semantics. We introduce encoder-wide directional consistency, a design principle that coordinates the modeling of directed computational semantics across node initialization, global spectral encoding, pairwise interaction, and feature updates. To implement this principle, we propose DTA-GT, a Direction- and Topology-Aware Graph Transformer that integrates direction- and topology-aware node initialization, Magnetic Laplacian spectral encoding, direction-dual structural attention, and a Direction-Sensitive MoE-FFN. Across accuracy and latency prediction on NAS-Bench-101 and NAS-Bench-201, DTA-GT consistently outperforms representative sequence-, GNN-, Transformer-, and hybrid predictors under limited-budget settings. Ablations and Magnetic spectral controls support the contribution of the direction-aware stages and orientation-sensitive spectral information, while mechanistic diagnostics characterize module behavior. These results support encoder-wide directional consistency as an effective design principle for architecture DAG representation.
| Comments: | Accepted at NeurIPS 2026. 38 pages, 6 figures |
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Neural and Evolutionary Computing (cs.NE) |
| Cite as: | arXiv:2610.16822 [cs.LG] |
| (or arXiv:2610.16822v2 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2610.16822 arXiv-issued DOI via DataCite |
Submission history
From: Yuxiang Zeng [view email]
[v1]
Mon, 12 Oct 2026 14:04:21 UTC (629 KB)
[v2]
Wed, 4 Nov 2026 11:26:03 UTC (1,268 KB)
来源：arXiv cs.AI · arxiv.org