AI 智能体如何基于视觉数据做决策？
How Do Artificial Agents Make Decisions in Light of Visual Data?
AI 导读
一项发表于 IEEE VIS 2026 GenAI 与智能体研讨会的研究分析了 VLM 智能体基于视觉数据的决策过程，通过两项实验评估其对可视化中不确定性的感知与决策表现。结果显示，GPT-4o 在饼图和柱状图的不确定性感知中准确率最高；当不确定性以圆形图、图标阵列或文本呈现时，自主智能体的决策最符合理性决策理论。
正文 · 原文
Abstract:Vision-language models (VLMs) have demonstrated significant advances in reasoning with visual data, which is particularly important given that much of human knowledge is encoded visually in scientific articles and other sources. However, a gap remains in understanding how artificial agents, powered by VLMs, perceive and navigate decision-making under uncertainty in the presence of visual data. This study explores this gap by evaluating and analyzing autonomous decision-making processes. Building upon prior work in visualization literacy and decision theory, we conducted two complementary studies: one examining how agents perceive and quantify uncertainty encoded in visualizations, and another investigating how artificial agents make decisions based on visual data in the form of images. We analyzed perception accuracy and decision performance across models and visualization types. Our results show that GPT-4o demonstrates the highest accuracy in perceiving uncertainty in pie and bar charts among the tested models. For decision-making, autonomous agents showed the best alignment with rational decision theory when uncertainty was represented through circle charts, icon arrays, or text.
| Comments: | Presented at the Workshop on GenAI, Agents, and the Future of VIS, held in conjunction with IEEE VIS, 2026 |
| Subjects: | Artificial Intelligence (cs.AI); Human-Computer Interaction (cs.HC) |
| Cite as: | arXiv:2611.05671 [cs.AI] |
| (or arXiv:2611.05671v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2611.05671 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Shayan Monadjemi [view email]
[v1]
Tue, 3 Nov 2026 21:58:10 UTC (609 KB)
来源：arXiv cs.AI · arxiv.org