基于高次认知的同步视听觉脑机接口字符拼写范式研究

Brain-computer interface technology has broad application prospect in the field of elderly care, disabled assistance, rehabilitation, hazardous environment operation, military and so on. However, the brain-computer interface technology is unable to meet the needs of practical application because of low accuracy and information transfer rate. Firstly, we investigate the neural mechanisms of higher cognition of simultaneous audiovisual stimuli and ascertain neural mechanisms and EEG change of modulation of spatial-semantic consistency or emotion on the processing of audiovisual information. On the basis of these, a region-based spelling paradigm for brain-computer interface system is constructed based on the higher cognition of spatial-semantic consistency or emotion. Then, we set up a platform of brain-computer interface system to verify the validity of the proposed spelling paradigm of brain-computer interface system through comparative research. In the end, a P300 -based brain-computer interface spelling system with high information transfer rate and low error rate is constructed. This project provides the theory and technical support for commercialization of brain-computer interface technology, which is of great significance to improve people’s quality of life and promote the development of brain-computer interaction technology.

脑机接口技术在老年人护理、残疾人辅助、医疗康复、危险环境作业、军事等领域具有广阔的应用前景，但目前的脑机接口技术面临着识别正确率不高和信息传输率低等问题，无法满足实际应用的需要。本项目拟从研究视听觉刺激的高次脑认知机制出发，从本质上弄清空间语义一致性、情绪及其一致性对视听觉信息加工的影响脑机制及其脑电表征。在此基础上，结合区域闪刺激诱发范式，构建基于空间语义一致性和情绪认知的同步视听觉脑机接口字符拼写范式。然后，搭建脑机接口系统平台，通过比较研究，验证新脑机接口字符拼写范式的有效性。最终构建一个信息传输率高、误差率低的基于P300电位的脑机接口“意念”字符拼写系统。本项目的实施为推动脑机接口技术走向产品化和商业化提供了理论和技术支持，对提高人们的生活质量，促进人机交互技术的发展具有重要意义。

本项目研究基于高次认知的同步视听觉脑机接口范式，项目的预期目标是通过研究空间语义、情绪及其一致性对视听觉信息加工的影响脑机制及其诱发脑电表征，提出一种基于空间语义和情绪的同步视听觉脑机接口字符拼写范式，增大特征脑电的振幅，提高脑机接口系统的分类准确率和信息传输率。围绕目标项目进行了针对性的研究，首先从本质上揭示了空间语义一致性、情绪及其一致性对视听觉信息加工的影响脑机制；在此基础上提出了基于空间语义一致性的视听觉双模态P300字符拼写范式，与单一视觉范式相比提高了字符拼写系统的性能；构建了基于情绪的视听觉双模态P300字符拼写范式，进一步优化字符拼写系统的性能；提出时变脑网络分析方法，揭示了在视听觉P300拼写范式下刺激诱发电位的深层神经机制；探索结合注意机制、自我面孔、熟悉面孔的字符拼写范式并比较新范式的性能；提出了新的卷积神经网络(CNN)分类方法，进一步优化基于传统CNN的P300脑电信号检测方法。本项目的研究成果将在老年人和残疾人对外交流、医疗康复、危险环境作业、军事等多个领域具有广阔的应用前景。