arXiv cs.AI· Wentao Ye, Yali Ye, Zhiqing Xiao, Ru Peng, Xinpeng Ti, Liyao Li, Zhanming Shen, Peng Lu, Jie Zhang, Haobo Wang·· 3 小时前AI 评分40
StruPO：从结构反馈到提示词策略，学习忠实的文生图提示词编辑器
From Structural Feedback to Prompt Policies: Learning Faithful Text-to-Image Prompt Editors
AI 导读
研究者提出 StruPO，一种结构感知的文生图提示词编辑方法，利用校准场景图作为策略条件与图像监督的共享表示，通过删除、重排和插入实现 token 级编辑。实验覆盖三个 2K 提示词测试集和四个冻结生成骨干，与学习型优化器、扩散模型微调及推理时注意力控制对比，StruPO 提升了忠实度—美观权衡，关系保真度持续提高，并经 15 人标注研究验证。
正文 · 原文
Abstract:Prompt optimization should improve the image a user receives without changing the scene the user requested. Yet stylistic rewriting can weaken object attributes and relations even when global text--image similarity remains high. We propose StruPO, a structure-aware prompt editor that uses a calibrated scene graph as a shared representation for policy conditioning and image-grounded supervision. The graph exposes what should be preserved, while deletion, reordering, and insertion provide a token-aligned way to modify the prompt. This turns structural feedback from an external critique into a training signal for a reusable editing policy. Learning from that signal requires handling both uncertain grounding and stochastic image generation. StruPO retains grounding confidence through smooth reward shaping and averages multiple images for each edit before computing group-relative advantages. Experiments span three disjoint 2K-prompt test sets and four frozen generation backbones, with comparisons to learned prompt optimizers, iterative structural feedback, diffusion-model fine-tuning, and inference-time attention control. StruPO improves the faithfulness--aesthetics trade-off, with consistent gains in relational fidelity. Independent automatic evaluators and a 15-annotator study provide complementary evidence, and human preference margins increase with relation complexity. Controlled ablations connect these gains to graph conditioning, local editing, and image-grounded learning. The resulting policy produces reusable optimized text in one pass after graph calibration, without an iterative critique--rewrite loop.
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.36146 [cs.AI] |
| (or arXiv:2610.36146v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.36146 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Wentao Ye [view email]
[v1]
Mon, 26 Oct 2026 18:50:50 UTC (9,030 KB)
来源：arXiv cs.AI · arxiv.org