自然交互下人脑对言语信息处理的动态脑网络机制研究

Exploring the neural mechanism of speech information processing in the brain during natural interaction is significant for the improvement of early diagnosis and intervention of speech dysfunction in brain diseases, and also the development of brain-inspired speech analytical technology and artificial intelligence. In this project, we design two experiments of speech perception and speech production under the background of natural interaction, combine the characteristics of high spatial resolution of functional magnetic resonance imaging (fMRI) with the advantage of high time resolution of electroencephalogram (EEG) source reconstruction and source information flow analytic technique, make use of dynamic brain network modeling idea, to investigate the dynamic characteristics of brain region, brain connectivity among large-scale networks, and brain connectivity within the core sub-network from the spatial and temporal domains simultaneously. Finally, through the joint analyses of speech perception and speech production processes, we intend to reveal the spatiotemporal dynamic mechanism of brain network during speech information processing. In this project, we use the spatiotemporal combination and dynamic analysis as technical features, and investigate the speech interaction procedure from the perspective of network, breaking through the limit of left brain idea in previous cognitive models. The results are expected to promote the cognitive understanding of speech interaction, and provide referential theory of cognitive mechanism for rehabilitation of speech dysfunction in brain diseases and advance in human-computer interaction technology.

探索自然交互下言语信息处理的脑神经机制，不仅有利于提高言语功能障碍脑疾病的早期诊断和干预，对人机交互中类脑语音分析技术的发展和人工智能的进步也具有重要意义。本项目以自然交互为背景，以言语感知和言语产生过程为两个分支设计认知实验；结合功能磁共振成像高空间分辨率的特点，发挥脑电溯源-源信息流分析技术高时间分辨的优势，基于动态脑功能网络建模思想，从空间域和时间域同时探究自然交互过程中脑区活动、全脑大尺度网络间连接、核心子网络内脑连接的动态变化特性。最后，通过对言语感知和言语产生过程联合分析，深入解析人脑对言语信息处理的时空动态脑网络机制。本项目以时空联合和动态分析为技术特色，以网络的观点探究言语交互过程，突破以往以左脑为主的认知模型限制，研究成果有望促进对言语交互过程在认知层面的理解，以期为言语功能障碍脑疾病的康复和人机交互技术的进步提供可借鉴的认知机理理论。

探索自然情景下言语信息处理的脑神经机制，不仅有利于提高言语功能障碍脑疾病的早期诊断和干预，对人机交互中类脑语音分析技术的发展和人工智能的进步也具有重要意义。本项目基于自然刺激范式设计了言语感知和言语产生脑认知实验，探究自然言语感知和产生过程的脑神经动态机制。建立了一套信噪比优良的脑神经数据库;提出了基于多被试超对齐的脑电-语音神经耦合建模算法，面向自然刺激的脑状态建模方法和脑功能网络构建方法；揭示了自然连续言语感知和言语产生过程的关键脑区、大尺度脑网络,核心子网络及其动态交互过程；发现无论是言语感知还是言语生成过程都受到高级认知自上而下的调控，这些调控与内在的语言学知识、当前的实验任务以及目标预期有关；最后,构建了言语生成与感知的神经功能模型，将言语生成与感知整合在统一的预测编码框架。本项目以时空联合和动态分析为技术特色，以网络的观点探究了言语交互过程，突破了以往以左脑为主的言语认知模型的限制，加深了我们对自然情景下语音信息处理机制的理解。