基于信息融合的多模态脑机接口及其应用研究

Brain computer interfaces (BCIs) may provide an alternative communication method for patients, because they can detect intention-specifc changes in brain signals. However, as cognitive abilities of patients with disorders of consciousness (DOC) might be much lower than those of the healthy subjects, the performance of BCIs for these patients is generally poor. This project will study multimodal brain computer interface system based on information fusion and applications for DOC patients. The proposed project focus on the optimization of information fusion algorithms, the implement of multimodal BCI based on information fusion and clinical applications. In this project, 1) Study information fusion algorithms as the core of the brain signal analysis algorithms, to provide support for the online BCI system; 2) Establish the BCI paradigm using two or more multimodal stimulation and the multimodal feedback shall be implemented by incorporating virtual reality and visualization technologies, then the multimodal BCI based on information fusion shall be accomplished; 3) We will develop a multimodal BCI for DOC patients and realize the clinical application of the brain computer interface into the field of functional assistance for DOC patients. This project can provide a new communication channel for patients with complete locked-in syndrome, and has with significant theoretically value and practically significance.

脑机接口技术通过检测脑电信号来判断用户的意图，可以给患者提供一种新的沟通渠道。但是由于意识障碍患者的认知水平远低于正常人，目前应用于此类患者的脑机接口性能较差。本项目将重点研究基于多模态信息融合的脑机交互系统及其对意识障碍患者的应用。本项目从信息融合算法优化、基于信息融合的多模态脑机接口系统构建以及临床应用三个方面展开工作：1）研究基于信息融合的脑电信号分析算法，为脑机接口在线系统提供算法支持；2）建立两种或两种以上的多感官刺激范式，结合虚拟现实与可视化技术构建多模态神经反馈，来构建基于信息融合的多模态脑机接口系统；3）开发面向意识障碍患者的脑机交互系统，实现脑机接口在意识障碍患者临床功能辅助领域的应用。本项目的研究可以给完全闭锁综合征患者提供一种新的沟通渠道，具有重大的学术价值和重要的社会意义。

脑机接口技术提供了一种不依赖于正常神经肌肉传导通道的交流与控制方式，是神经工程领域领先发展、重点研发的前沿热点。本项目针对基于信息融合的多模态脑机接口及其应用领域中的一些关键问题开展研究：1）提出了新的或者改进的基于深度卷积神经网络的多种脑机接口信号特征提取和分类算法以及基于小训练样本集下迁移分类算法，在减少受试者训练时间的同时，提高系统的准确率和信息传输率；2）研究整合视听通道刺激的多模态刺激范式，设计了融合P300和SSVEP的异步多模态脑机接口范式和系统，并建立了基于P300在线脑机接口游戏系统，更进一步扩展了应用领域；3）研发面向临床应用的多模态脑机接口系统，开展面向意识障碍、抑郁症等患者的辅助诊断研究。本项目的研究成果可以给意识障碍等患者提供一种新的沟通和诊断方式，具有广阔的应用前景。