面向多模态概念获取和绑定的类脑层级结构神经网络研究

Acquiring and binding concepts from the multi-modal stimuli is one of the most important abilities of the brain as well as the cornerstone of human cognitive ability. Therefore, it is of great significance for the brain-like intelligence to build a neural network model for multi-modal concept acquisition and binding. The current main stream study of the artificial neural network field seldom considers the concept acquisition of multiple sensory channels and concept binding among different sensory channels. It is the absence of the theoretical research in the field of artificial neural network and brain-like intelligence. In order to make up this deficiency and based on some latest research results of some related disciplines (including brain science, neural computing, and so forth), this project proposes a research on brain-like hierarchical neural network model for multi-modal concept acquisition and binding, and the project is mainly to achieve the following goals: (1) based on the modularized structure and the hierarchical information processing procedure of the brain, establishing the framework of the network, designing the structure and function of different sensory modules, and the association mode among different sensory modules. (2) studying the concept organization mode and the way of the concept acquisition in different sensory pathways, designing particular concept acquisition algorithms for particular sensory channels. (3) studying the multi-modal concept binding function of the association areas of the brain and designing multi-modal concept binding algorithm which can be applied to online learning environment.

从多模态刺激中获取概念和绑定概念是大脑最为重要的能力之一，也是人类认知能力的基石。因此，建立关于多模态概念获取和绑定的神经网络模型对类脑智能意义重大。而目前主流的人工神经网络研究较少综合考虑多个感官通道的概念获取和不同感官通道间概念的绑定，这是人工神经网络和类脑智能研究领域一个理论上的缺失。为弥补这一缺失，本项目在广泛调研了当前相关学科（如脑科学、神经计算科学等学科）最新研究成果的基础上，提出面向多模态概念获取和绑定的类脑层级结构神经网络研究，主要实现以下目标：（1）以大脑的模块化组织结构和层级式的信息处理流程为基础，规划神经网络的总体框架，设计不同感官模块的结构、功能以及各感官模块之间的联系模式。（2）研究不同感官模块感觉传导路径中概念的组织模式和概念的习得方式，设计相应的概念获取算法。（3）研究大脑联合区域关于多模态概念绑定的功能，建立一个适用于在线学习环境下的多模态概念绑定算法。

从多模态刺激中获取概念和绑定概念是人类认知能力的基石。建立关于多模态概念获取和绑定的神经网络模型对类脑智能领域意义重大。在上述背景下，本项目提出了面向多模态概念获取和绑定的神经网络研究。主要研究内容包括：（1）面向多模态概念获取和绑定的通用型神经网络总体结构研究；（2）不同类型感官通道的概念获取算法研究；（3）适用于多感官通道的概念绑定算法研究。在项目实施过程中，课题组研究人员在广泛调研近些年关于大脑结构研究的基础上，建立了一个类脑层级式的神经网络框架；设计了视觉、听觉和味觉通道的概念获取算法；形成了一种面向多模态的“内省+交互+学习”式的概念绑定算法。项目完成高质量论文5篇。