上肢运动意图的脑电深度学习识别与任务导向递归贝叶斯估计解码研究

The rehabilitation robot combined with non-invasive brain-computer interface (BCI) can provide the upper-limb motor function rebuilt/maintenance for hemiplegic patients/astronauts. However, the conventional BCI can only recognize the motor imagery of different limbs from electroencephalographic (EEG) signals in non real-time for controlling the upper-limb rehabilitation robot, leading to a heavy latency between movement/motor imagery initiation and feedback perception, as well as the non-intuitive human-robot interaction, which will severely affect the effectiveness of rehabilitation therapy. In order to overcome the above drawbacks of the conventional BCI, this project will research on the recognition methods of the discrete movement intention before the voluntary movement/motor imagery of the same upper-limb in order to offer the timing margin for the feedback devices; as well as the real-time decoding methods of the continuous movement intention during the voluntary movement/motor imagery of the same upper-limb so as to provide a friendly human-robot interaction control interface. To this end, (1) this project will determine the EEG signal representations of the discrete movement intentions for the upper-limb through deep learning networks with similarity constraints-based pre-training and robust bimodal deep fusion; (2) this project will reveal the rules for ensuring that the output of the EEG signal decoder satisfies the particular movement control requirements of the rehabilitation training task via the task-oriented recursive Bayesian estimation. The outcome of this project shall greatly improve the recognition accuracy of discrete movement intentions before the voluntary movement/motor imagery of the same upper-limb, as well as the reliability of the EEG signal decoder that outputs continuous movement control instructions required by the rehabilitation training task for the same upper-limb, providing the theoretical and technical supports for developing the BCI system in neuro-robot for the upper-limb motor function rebuilt/maintenance.

康复训练机器人结合无创脑机接口技术为偏瘫患者/宇航员上肢运动功能重建/维持提供了可能。传统的脑机接口从脑电信号中非实时地识别出不同肢体的运动想象再间接控制上肢康复机器人，导致启动运动-感知反馈时延较大，人机交互不直接自然，严重降低康复治疗效果。为克服上述局限，本项目拟研究同一上肢自发运动/运动想象开始前离散运动意图的提前识别，以向反馈装置提供时间余量，及开始后连续运动意图的实时解码，以提供友好的人机交互控制接口，重点在于：（一）通过相似性约束预训练的深度网络、双模鲁棒深度融合探明上肢离散运动意图在脑电信号中的表征；（二）通过任务导向递归贝叶斯估计揭示使得脑电信号解码器输出训练任务所要求运动的规律。本项目的研究将极大地提高同一上肢自发运动/运动想象开始前识别离散运动意图的准确性，以及开始后解码连续运动意图的可靠性，为研发面向上肢运动功能康复/维持的神经-机器人脑机接口系统提供理论和技术支撑。

针对助老助残、军事航天等领域脑-机器人交互任务的自然直观、灵活易用、稳定鲁棒等交互需求，本项目开展了基于同一上肢自发运动/运动想象脑电信号的脑机接口及脑机交互基础问题和关键技术的研究。（1）研究了基于脑电信号的同一上肢离散运动意图识别方法。针对同一上肢运动准备/运动想象脑电信号的复杂统计特性，基于深度学习、稀疏学习、脑连接网络等手段探明了上肢离散运动意图在脑电信号时-频-空间域的层次化非线性特征表现，在此基础上建立了基于脑电信号的同一上肢离散运动意图识别模型。（2）探讨了基于脑电信号的同一上肢连续运动意图解码方法。首先提出了脑电信号时频复数值和脑网络连接的连续运动意图表征方法，实验结果表明以这些表征与常用的脑电信号幅值表征相比，对噪声更鲁棒，以其作为递归贝叶斯估计或多元线性回归解码模型输入，统计显著地改善了以脑电信号幅值表征为输入的解码模型泛化性能；其次针对目前常用的卡尔曼滤波或多元线性回归解码模型无法精确反映脑电信号表征与连续运动意图之间复杂的非线性动力学特性，提出了复数值深度循环神经网络的解码模型，进一步提升了基于脑电信号的连续运动意图解码精度。（3）探索了基于混合脑电-视线人机接口的意图输入。针对脑机交互任务中仅使用脑电信号无法满足意图输入灵活性、鲁棒性需求的不足，在已实现的基于脑电信号的意图输入基础上，探索了基于视线的意图输入，并实现了脑电、视线双模态鲁棒融合的指令输入。（4）搭建了基于混合脑电-视线人机接口的人-机-环交互系统。在上述研究的基础上，进一步开展了机械臂/多智能体的半自主遥操作控制、多模态感知反馈的研究，搭建了集成脑电、视线等输入模态以及视觉、力触觉反馈模态的机械臂/多智能体半自主遥操作控制实验平台，实现了“脑-机-环-脑”双向闭环的脑-机交互，提高了交互的自然直观性、灵活易用性、稳定鲁棒性以及交互效率。