一种常人参与的脑机融合方法及其在机器舞蹈中的应用

Brain-machine integration has increasingly become an essential approach to human-machine hybrid intelligence and a breakthrough in traditional artificial intelligence. Most researches so far rely on invasive destructive methods applied to animals and the disabled people, rather than human-machine interaction engaging normal people. Such methods cannot be applied in human-machine interaction studies engaging normal people.. In order to move beyond the limitation in the application of current brain-machine integration, this study will introduce a brain-machine integration method engaging normal people and apply it in the construction of a social robot dance system by adopting non-destructive brain technology, interpreting motor EEG signals interactively, and constructing action planning strategy based on reinforcement learning. This project will mainly involve the following:. （1）Interpretation of motor imagery EEG: We will analyze human motor imagery EEG signals through the construction of EEG pattern analysis algorithm and identify intention EEGs for moving the tongue, the left hand, the right hand and the feet respectively in order to plan for the robot to comply with such intention.. （2）Analysis of action observation EEG: This is used to judge and correct the movements of the robot. Since there is a difference in terms of brain area and sequence between imagery and observation, effective collecting and interpreting methods should be established respectively for each step, however, a comparative study between the two is to be implemented. . （3）Establishment of reinforcement learning for the brain-machine integration method: We will introduce reinforcement learning methods to use motor imagery intention, environment information perception and robot behavior judgement appraisal so that our computation method will implement real-time action planning, realize human-robot interaction, and improve the performance of the robot effectively. . （4）Construction a human-robot prototype system for dance: We will construct a prototype system for dance based on brain-machine integration method involving the above three major sub-projects. In addition, the robot shall also be programed to move its body according to specific music so that both environment music-driven and human intention-induced movements will combine to produce the optimal performance. . In effect, we aim to design a new brain-machine integration computing method to enable nondestructive EEG to be used and analyzed for real-time interaction between a normal person and robot. Obviously, our study goes beyond the limits of traditional artificial intelligence, treats the issue of motor observation EGG interpretation, and will make a big contribution to brain-machine integration by presenting a new computation method and having it applied in a real system for robot dance.

脑机融合日益成为实现混合智能的重要方法，也是目前突破传统人工智能局限性的新途径。但在迄今为止已有的初步研究中，由于难以摆脱侵入式有损大脑的探测手段，大多数脑机融合计算系统均局限于动物模型或残疾人辅助控制方面，难以运用到常规的人机交互系统构建之中。为了突破目前脑机融合应用范围的限制，本项目将以无损脑电技术的交互性解读为手段，来给出一种常人参与的脑机融合计算方法，并运用到社交型机器舞蹈系统的构建之中。具体研究内容包括：运动想象脑电意图解读、动作观察脑电评判解读、建立基于增强学习的脑机融合计算方法、以及构建人机交互舞蹈的脑机融合原型系统。这样的研究，有助于突破传统人工智能方法的局限性，特别是对于解决运动观察脑电的解读问题，对于探索一种适用常人参与的脑机融合计算新方法，对于构建社交型机器人舞蹈系统的新途径、以及对于拓展脑机融合应用新领域，无疑都有重要的科学意义和潜在的应用价值。

脑机融合可以为机器人及信息技术的进步提供新的契机，因此近年来，脑机融合的研究工作，引起了国际学术界的广泛关注。在这其中，一个关键又困难的课题，就是常人参与的无损脑机融合方法的机器实现。我们经过四年的研究工作，给出一种构建社交型机器人舞蹈系统的新方法，对于推动脑机融合方法及其在人机交互舞蹈研究的发展，做出了重要贡献。. 首先，为了解决制约运动想象脑机接口分类性能的两个挑战，我们引入深度RNNs模型，提取空域-频域-时域特征，完成对运动想象脑机接口系统的分类任务，并以此提升了对运动想象脑机接口系统的分类性能。. 其次，为解决传统强化学习效率低、环境要求高的缺陷，我们通过错误相关电位脑机接口实验范式，为强化学习系统提供偏好选择，极大的提升了强化学习效率和减轻了环境需求。采用训练好的R3DCNNs模型，通过在线实验范式完成偏好选择的分类，并训练强化学习系统，在错误相关电位脑机接口中为强化学习提供了31.67%的效率提升。. 然后，我们提出一种音乐驱动的机器人舞蹈表演系统，利用深度学习在音乐信息提取和序列建模方面的优势，解决现实数据集规模小，舞蹈风格多样的问题，同时保证机器人动作的稳定和流畅性，为脑机融合的机器乐舞系统的构建，提供了一个基本的框架。. 最后，我们利用人脑运动想象作为指导，采用强化学习方法，把工作评判和观摩学习作为不断完善机器舞蹈的依据，实现一种脑机融合的人机舞蹈智能系统。这样的脑机融合系统，可以使脑机两者互为适应、相互协作、融为一体，形成混合智能，真正超越原有机器智能的局限性。. 总之，研究项目所取得的主要进展和成果，无论在运动脑电信号分析方法的发展、无损脑机融合新方法的提出，还是在音乐驱动机器舞蹈系统的搭建方面，以及在一种脑机融合的人机舞蹈系统构建途径等方面，均给出了具有鲜明创新价值的研究结果。对于开拓基于无损脑机融合计算方法的机器实现，丰富人机交互性舞蹈机器人系统构建的新途径，无疑是有着重要的建设性意义。