PIP-ToM：面向多模态心智理论的视角有界证据校准方法
Who Knows What - and What Does It Support? Perspective-Bounded Evidence Calibration for Multimodal Theory of Mind
AI 导读
研究者提出 PIP-ToM，一种证据可采性接口，通过为各智能体构建专属证据存储、对齐结构化心理状态主张并校准贝叶斯假设评分，解决多模态心智理论任务中的视角泄漏和信息源混淆问题。PIP-ToM 结合 GPT-4o 在 MuMA-ToM、MMToM-QA 和 Hi-ToM 三个基准上超越现有 ToM 方法，平均准确率分别达到 92.56%、95.00% 和 95.00%。
正文 · 原文
Abstract:Theory of Mind (ToM), the ability to infer others' beliefs, goals, and intentions, is a cornerstone of social intelligence. Multimodal ToM requires inferring an agent's mental state from evidence that is available to that agent, rather than from the complete episode observed by the solver. Existing LLM-based and Bayesian model-based approaches reconstruct global event traces, yet they do not explicitly distinguish objective events from agent-admissible evidence during hypothesis evaluation. Consequently, agent-inaccessible events or semantically misaligned actions and utterances may influence the inferred belief, goal, or social intention, leading to perspective leakage and source or route confusion. We formulate this limitation as an evidence-admissibility problem: evidence should contribute to a mental-state hypothesis only when it is accessible to the relevant agent, preserves its information source, and is diagnostic of the queried mental-state variable. To operationalize this principle, we propose PIP-ToM, an evidence-admissibility interface that constructs agent-specific evidence stores, aligns them with structured mental-state claims, and calibrates model-based hypothesis scores while preserving the underlying Bayesian inference process. PIP-ToM with GPT-4o outperforms existing ToM methods across MuMA-ToM, MMToM-QA, and Hi-ToM, achieving average accuracies of 92.56%, 95.00%, and 95.00%, respectively.
| Subjects: | Artificial Intelligence (cs.AI) |
| Cite as: | arXiv:2610.36824 [cs.AI] |
| (or arXiv:2610.36824v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2610.36824 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Ziqi Liao [view email]
[v1]
Tue, 27 Oct 2026 08:23:46 UTC (11,728 KB)
来源：arXiv cs.AI · arxiv.org