内在目标驱动的自主智能体：分层探索捕捉果蝇未来自发动作
Intrinsic Goals for Autonomous Agents: Hierarchical Exploration Captures Future Volitional Action in Drosophila
AI 导读
一项研究将果蝇全脑记录与神经力学数字孪生结合，测试优化内在目标能否产生与果蝇行为匹配的具身智能体。团队用 20 种内在动机算法（涵盖好奇心、熵和 competency 方法）在无外部奖励下训练智能体，其中 DIAYN 这一分层 competency 算法在行为和神经前瞻编码上与果蝇吻合最佳，其策略还能捕捉果蝇探索策略的个体差异及半球间运动学编码的加减规则。
正文 · 原文
Abstract:Animals have a remarkable capacity for autonomy---driven by intrinsic motivation---without external sensory cues or extrinsic objectives, yet the computational mechanisms behind this ability largely remain a mystery. We combine whole-brain recordings in the fruit fly (Drosophila melanogaster), which reveal neural representations of spontaneous exploratory decisions more than 10 seconds in advance, with a neuromechanical digital twin to test whether optimizing intrinsic goals yields realistically embodied artificial agents that quantitatively match fly behavior and internal representations of future volitional action during exploration. Our hierarchical policy selects from a discrete library of low-level motor primitives, consistent with ethological theories of behavioral syllables. We train agents without external reward using 20 intrinsic motivation algorithms spanning curiosity, entropy, and competency methods. The agents are evaluated in a virtual tethered-ball assay modeled on the fly experiment, comparing spontaneous locomotor statistics and internal representations and decoding future turn direction from kinematics and neural activity on the same timescales observed in flies. DIAYN, a hierarchical competency algorithm learning discrete policies that maximize behavioral entropy and the separability of induced sensory observations, achieves the strongest joint behavioral and neural prospective coding performance relative to flies. Its learned policies closely capture individual variability in fly exploration strategies, suggesting that this diversity serves to maximize state-space coverage at the population level. DIAYN's latent activity also closely captures the sum-difference rule for interhemispheric encoding of kinematics in biological flies. These results establish hierarchical, intrinsically motivated agents as a powerful framework for reverse-engineering animal autonomy.
| Comments: | 27 pages, 5 figures |
| Subjects: | Neurons and Cognition (q-bio.NC); Artificial Intelligence (cs.AI); Machine Learning (cs.LG); Robotics (cs.RO) |
| Cite as: | arXiv:2610.34638 [q-bio.NC] |
| (or arXiv:2610.34638v1 [q-bio.NC] for this version) | |
| https://doi.org/10.48550/arXiv.2610.34638 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Reece Keller [view email]
[v1]
Sun, 25 Oct 2026 23:40:13 UTC (509 KB)
来源：arXiv cs.AI · arxiv.org