视觉-前庭觉整合与运动控制的一体化仿脑模型

In the highly dynamic environment, the brain forms flexible and reliable perception by integrating information conveyed through multiple sensory modalities, e.g., vision, audition, vestibular senses, etc., and thereafter executes robust motion control that optimizes the survivorship of itself. Extensive behavior and neural electrophysiological studies show that the brain integrates multisensory signals in a near-optimal way that is predicted by Bayesian inference. However, the neural architecture and computational mechanism underlying this feat is largely unclear. This project will study visual-vestibular integration as an example of bimodal integration. We will first build a unified biological neural network models for the two scenarios based on a decentralized architecture supported by neural electrophysiological data. Furthermore, simulated sensory inputs (binocular optic flow and acceleration signals) will be generated on virtual robotics platforms. Combining brain-inspired decentralized network model and active efficient coding framework, we will build a data-driven network model that unifies the learning of sensory representation and motion control. The properties of the neurons in the network model trained with simulated data will be compared with the firing patterns of real neurons, providing insights about the interaction of multisensory information on different levels in the brain. This project integrates the frontiers in the brain science and robotics, and contributes to the theoretical foundation of the design of next-generation bio-inspired robots.

在复杂多变的环境中，大脑通过视觉、听觉、前庭觉等多种感觉提取信息，整合形成了灵活而又可靠的感知，为其做出最有利于生存的运动控制提供了基础。大量行为学以及神经电生理学研究显示，大脑的多感觉整合近似符合贝叶斯统计理论预测的最优。然而，其具体神经网络结构和神经计算机制仍然并不清晰。本研究将以视觉-前庭觉整合为例，基于解剖学和神经电生理数据，首先建立双感觉在时间和空间上整合的生物学神经网络模型。进一步的，借助虚拟机器人仿真平台生成多感觉信号（双眼光流以及加速度信号），将仿脑的分布式网络结构与主动有效编码的学习框架相结合，试图建立数据驱动的从原始多感觉信号的表征到运动控制的一体化网络学习模型。通过对比真实神经元数据与训练所得的神经网络特性，将深化理解多感觉信息在大脑不同层级的相互作用。本研究结合脑科学与机器人学的研究前沿，将为下一代仿生机器人的制造提供理论依据。

大量行为学以及神经电生理学研究显示，大脑的多感觉整合近似符合贝叶斯统计理论预测的最优。本研究以视觉-前庭觉整合为例，探索了多感觉整合的具体神经网络结构和神经计算机制。.首先，我们基于神经电生理和解剖学数据，建立了统一的生物学神经网络模型。其次，我们在仿真平台上验证了数据驱动的从原始多感觉信号的表征到运动控制的一体化网络学习模型。我们在机器人目标检测以及仿脑声源分离任务上，验证了该仿脑架构的优越性，并在物联网芯片上探索了产业化的可行性。本研究对揭示大脑的多感觉整合机制做出了一定贡献，同时有望为下一代仿生机器人的感知和控制系统提供理论依据。