XStack：一个用于 ML 代码库迁移的基准测试
XStack: A Benchmark for ML Repo Migration
AI 导读
arXiv 论文提出 XStack 基准，评估 AI 智能体将 PyTorch 代码库迁移为可在 Google TPU 上运行的 JAX 代码库的能力。结果显示迁移仍不可靠：Claude Opus 5 通过率 75%，GPT-5.6-sol 为 45%，其余智能体最高不超过 40%。XStack 已开源发布，目标是让跨框架与硬件的可靠迁移进展可衡量。
正文 · 原文
Abstract:ML code should run where researchers choose, not where it was first written. Today, machine learning code mainly lives in two ecosystems, PyTorch on NVIDIA GPUs and JAX on Google TPUs. Migrating repositories between them means building and testing across frameworks and hardware. This migration is (1) a practical need, since it unlocks different hardware for researchers, and (2) a natural benchmark: it involves different frameworks and hardware, covers whole repositories, and is long-horizon and challenging. We introduce XStack, a benchmark that evaluates agents on migrating ML repositories between these two ecosystems. Each agent receives a PyTorch repository and a clean TPU machine and must deliver a JAX repository. Its output passes if it meets our grading criteria, based on the source repository's published result, authenticated and calibrated by us. Migration by agents remains unreliable: Claude Opus 5 passes 75%, GPT-5.6-sol 45%, and every other agent at most 40%. XStack is a step toward freeing ML code from its ecosystem. We release XStack to make progress toward reliable migration across frameworks and hardware measurable.
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2611.08472 [cs.LG] |
| (or arXiv:2611.08472v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2611.08472 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Linrong Cai [view email]
[v1]
Thu, 5 Nov 2026 18:55:54 UTC (367 KB)
来源：arXiv cs.AI · arxiv.org