JEPA-PS 如何从奖励无关预训练中学习黑盒优化
JEPA as a Transferable Optimizer: Learning and Understanding Black-Box Optimization from Reward-Agnostic Pretraining
AI 导读
研究者提出基于联合嵌入预测架构的 JEPA-PS 学习优化器,用随机 rollout 中参数向量与标量回报做预训练,测试时读取短评估历史、一次前向传播即提出候选解,无需任务描述、奖励塑形或测试期梯度。
正文 · 原文
Abstract:Black-box optimization evaluates candidates through simulations, rollouts, or experiments, often without gradients. Learned optimizers are usually judged by whether they outperform existing methods. We ask a complementary question: what information does a learned optimizer use from its search history? We introduce JEPA-PS (JEPA Policy Search), based on a Joint-Embedding Predictive Architecture. It is pretrained on random rollouts using parameter vectors and scalar returns. Its latent-prediction objective predicts the next point in an optimization trajectory rather than its reward or gradient, allowing us to examine what was tried and how well it performed. At test time, JEPA-PS reads a short evaluation history and proposes a candidate in one forward pass, without task descriptions, reward shaping, or test-time gradients. We evaluate it on 15 control environments against 12 optimizers, including evolutionary methods, Bayesian optimization, CMA-ES, and an LLM optimizer. JEPA-PS is competitive with tuned classical baselines and outperforms the LLM optimizer by more than 2x on Swimmer at 71x lower per-step cost. It also transfers zero-shot to environments excluded from pretraining. Null-model comparisons and decoder-training ablations indicate that its latent representation carries information beyond simple history statistics; every trained decoder variant performs worse than the default frozen representation. Our main finding is that the history JEPA-PS uses depends on the task. On smooth landscapes it uses the full trajectory, whereas on rugged landscapes the best-seen point carries nearly all the signal. This memory requirement has a weak positive correlation with fitness-distance correlation (FDC; r=0.112, n=7), while an independent reward-ordering sensitivity probe does not corroborate the relationship (r=-0.024, n=7). Code: this https URL.
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2611.23608 [cs.LG] |
| (or arXiv:2611.23608v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2611.23608 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Manan Roy Choudhury [view email]
[v1]
Tue, 17 Nov 2026 23:01:25 UTC (1,705 KB)
来源:arXiv cs.AI · arxiv.org