多时间尺度递归神经网络动作学习算法研究

Recently, the study of recurrent neural network (RNN) has made important progress in many fields, which greatly promotes the research and development of RNN in the field of behavior learning. Behavior learning is the key theoretical foundation for enabling robot arm's adaptive behavior. Considering the multi-timescale characteristics contained in the action sequence, this project intends to study the multi-timescale continuous recurrent neural network (CTRNN), design the optimization behavior learning algorithm and analyze the dynamic properties of the model. Firstly, in order to reflect the multi-timescale feature, a hierarchical network structure containing neurons encoding different speeds ("fast" and "slow") in action sequences will be constructed, the cost function is designed and an optimal learning algorithm is proposed. Secondly, based on the existence theorem of Nash equilibrium, the adaptive regulation rules of neuron connection weights will be presented and convergence of network is analyzed. Next, the learning algorithm of CTRNN under the noise condition is designed, and the robustness and the generalized learning ability are analyzed. Finally, the learning ability of network and theoretical effectiveness are verified using the robotic arm hardware platform. The above problems are more difficult and challenge in behavior learning by using CTRNN. Solving these problems will help to provide the important theoretical and experimental foundations to realize the autonomous learning ability for the robotic arms.

近年来，递归神经网络在很多领域取得了重要进展，促进了递归神经网络在动作序列学习领域的研究和发展,而动作序列学习是实现机器人手臂具有自适应行为的关键理论基础。考虑动作序列包含的多时间尺度特性，本项目拟研究多时间尺度连续递归神经网络（CTRNN）,设计优化动作学习算法并对模型的动力学特性进行理论分析。首先，为了体现多时间尺度特性，将构建包含体现动作序列中不同速度(“快”和“慢”等)的神经元的层级网络结构, 并提出优化学习算法；其次基于Nash平衡点存在理论，建立神经元连接权之间的自适应调节规则，进行收敛性分析；接着设计网络在噪音条件下的学习算法,进行鲁棒性分析，并对泛化学习能力进行分析；最后，搭建机器人手臂实验平台测试算法和理论的有效性。上述问题的解决将建立更加优化的动作序列学习算法，同时也为算法在实现机器人手臂对新任务的自主学习能力提供重要的理论和实验基础。

本项目面向递归神经网络的动作学习算法进行了深入探索，建立了优化的动作学习算法，同时也为算法在机器人上的应用提供了重要的理论和实验基础。本项目的主要工作进展及研究成果主要包括：构建多时间尺度递归神经网络结构并对动作学习算法展开研究，提出基于多时间尺度脉冲神经网络的小样本动作学习、基于网格细胞的递归神经网络的动作学习方法及混合架构强化学习的连续动作学习方法；对网格细胞编码的多尺度扩展与模型进行研究，对网格神经元之间的递归连接权重的参数化机制进行了优化，从而增强了路径积分模块在多尺度扩展上的易用性，提出基于多尺度网格神经元编码的路径积分和特征表示；研究了机器人手臂的动作学习问题，实现了基于全卷积神经网络的机器人抓取任务并提出了具有低功耗特点的脉冲神经网络的机器人抓取方法。项目期间共发表论文15篇，申请专利4项，研究成果具有较高的理论意义和实用价值。