基于视觉感知和深度学习的机器人鲁棒抓取研究

Aiming at the problem of uncertainty originating from modeling, perception and execution for robotic grasping under complicated environment, this project proposes to combine the biological mechanism of human vision system, introduce a multimodal deep learning method and study the robotic robust grasp theory and method based on visual perception and deep learning. A method for fast target recognition based on biomimetic visual properties is presented to address the object uncertainty during target recognition. In view of the perception uncertainty during robust grasp planning process, this project proposes a universal representation method of grasp configuration-“Polygonal pyramid” to express the entire grasping dimensions in 3D space; To solve the problem of similarity, diversity and overfitting in multimodal data learning, a grasp feature learning framework based on multimodal group regularization and a grasp configuration generation approach based on Gaussian pooling deep convolutional neural network are introduced to increase the accuracy and stability of grasp planning. According to the uncertainty in robust grasp execution, this project presents a selection algorithm for grasp control strategies based on Markov decision process and finally realize the grasp which gives the consideration to robustness and the execution speed. The implementation of this project is of both great theoretical and practical significance to build the robotic grasp theory based on visual perception and learning as well as increase the deep application of robots in the intelligent manufacturing field and so on.

本项目针对复杂环境下机器人抓取在建模、感知、执行过程中的不确定性问题，结合人类视觉生物机理，引入多模态深度学习方法，研究基于视觉感知和深度学习的机器人鲁棒抓取理论和方法。针对目标识别中的物体不确定性，提出类生物视觉特性的目标快速识别方法。针对鲁棒抓取规划过程中的感知不确定性，提出“多棱锥”抓取构型通用表示方法，实现对三维空间所有抓取维度的表达；建立基于多模态组调整的抓取特征学习框架和基于高斯池深度卷积神经网络的抓取构型生成方法，解决多模态数据学习中的相似性、差异性及过拟合问题，提高抓取规划的准确性与稳定性。针对鲁棒抓取执行中的不确定性，提出基于马尔科夫决策过程的抓取控制策略选择方法，最终实现兼顾鲁棒性和执行速度的抓取。本项目的实施，对于构建基于视觉感知与学习的机器人抓取理论，提高机器人在智能制造等领域的深度应用具有重要的理论与现实意义。

机器人抓取，尤其是非结构化环境下散乱未知多物体的抓取，一直是机器人领域具有挑战性的研究命题。本课题重点研究基于视觉感知和深度学习的机器人鲁棒抓取理论和方法。针对非结构化环境下散乱未知多物体的抓取检测问题，提出了基于完备抓取构型的两阶段抓取检测深度学习模型、包含遮挡关系的完备标签自动生成方法、基于ℋ散度理论的视觉域适应方法，建立了基于视觉域适应的抓取检测理论和方法。针对机器人在未知场景中智能抓取的三维物体理解问题，提出了鲁棒的特征点描述子对应、旋转等价卷积匹配、物体位姿估计及类别预测方法，建立了基于细粒度语义的三维物体理解方法。针对基于视觉感知的抓取难以评估稳定性问题，建立了基于视触觉融合的抓取稳定性评估方法，将触觉先验知识学习模型融入视觉抓取框架，实现机器人一次性高效稳定抓取。与上海发那科机器人有限公司开展产学研，开发了一套适用多品种、多形状的散装单零件或混装多零件的智能柔性分拣机器人系统，应用于电商智能分拣产线。