脉冲神经网络强化学习中的循环模块化结构：CoRe SNN 提出与验证
Recurrent Modular Structure in Spiking Neural Network Reinforcement Learning
AI 导读
研究者提出 Community Recurrent SNN（CoRe SNN），一类按固定角色划分为社区（控制器、动作、潜在状态）的循环脉冲神经网络，并引入新学习规则 Map-STDP 在学习中促进模块化结构。
正文 · 原文
Abstract:Biological neural circuits are both modular and recurrent, with dense communities sparsely linked to one another. Spiking neural networks (SNNs) for reinforcement learning are usually feed-forward, and it is unclear how to obtain functional modules in a recurrent SNN or what they buy. In this work, we introduce a class of spiking networks, called Community Recurrent SNNs (CoRe SNNs): recurrent networks partitioned into communities with fixed roles. A controller community receives the stimulus, each action has its own community, and latent communities carry internal state. Spiking activity is then a walk between communities, and the action is the community the walk reaches. All learning is local and three-factor, and the recurrent communities are closer to cortical microcircuits than a feed-forward actor. We compare three regimes for obtaining the communities: dynamic modularity, in which a new learning rule we introduce, Map-STDP, encourages modular structure during learning; partial modularity, in which a stochastic block model (SBM) initialization enforces it partly; and full modularity, in which sealing enforces it fully. On CartPole, across three spiking neuron models, Map-STDP improves learning over the same network without it matches full modularity (sealed communities) on two of the three models, and makes the observed spike walk more compressible. Replacing the feed-forward actor of a TD-STDP actor-critic with recurrent action communities improves on the feed-forward network on CartPole and LunarLander, and matches it on Acrobot. How strongly to enforce modularity depends on the task: full modularity helps most on Acrobot, while on LunarLander the dynamic, partial and full regimes perform alike. Overall, these recurrent modular networks can match or even improve upon feed-forward approaches, with improvements in bio-plausibility and energy efficiency.
| Comments: | 19 pages, 5 figures |
| Subjects: | Artificial Intelligence (cs.AI); Machine Learning (cs.LG); Neural and Evolutionary Computing (cs.NE) |
| Cite as: | arXiv:2611.14262 [cs.AI] |
| (or arXiv:2611.14262v1 [cs.AI] for this version) | |
| https://doi.org/10.48550/arXiv.2611.14262 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Vikram Ramavarapu [view email]
[v1]
Tue, 10 Nov 2026 16:09:19 UTC (899 KB)
来源：arXiv cs.AI · arxiv.org