人机交互合作装配中人体行为分析与理解方法研究

Flexibility and adaptability to rapidly changing market demands and product diversification is one of the highest design criteria for the assembly. Man-machine interaction cooperation assembly is an effective way to the demand. The action recognition and activity comprehension based on the sensors is the key technology in activity analysis. Action recognition and activity analysis are receiving increasing attention from the researchers in many their own fields. But it can't suit to assembly environment. This project would research on the action recognition and activity analysis method based on man-machine interaction cooperation in assembly. Firstly, the assembly environment information will be acquired based on the multi-vision intelligent space. The human action image feature selection and movement characterization methods in 3-D space will be analyzed. Secondly, the action pattern mode based on behavior dynamics will be constructed after the behavior pattern characteristic and representation method analyzed. Thirdly, the human body activity multilayer mode will be constructed. The contextual information in man-machine interaction cooperation assembly will be fused in the activity recognition. The comprehension mechanism between human complex activity，event and context will be revealed based on the dynamic system theory. At last, the activity will be recurred based on cognitive linguistics framework. The emergence behavior occurrence mechanism of behavioral dynamics system and evolution law will be elucidated. And a new method will be represented for behavior representation and activity analysis. The study of this project has important theoretical meaning and has a good application prospect in the common basic theories research field of multi-agent system, intelligent manufacturing system and complex adaptive system.

随着市场需求的多样化和多变性的加剧，对于生产装配的柔性和适应性提出了更高要求，人机合作装配为应对当前需求提供了有效方法，而基于传感数据来识别和理解人体动作和行为的行为分析是关键，但当前对于人体行为分析的研究只在各自应用领域，且不成熟，尚不能满足装配任务。本项目拟围绕人机交互合作装配中的人体动作识别、行为分析与理解方法展开，通过构建多目立体视觉装配智能空间，获取装配环境信息，研究三维空间的人体运动的图像特征选取和运动表征方法；研究行为模式的特征与描述方法，构建基于行为动力学的行为模式；构建多层次人体行为模型，融合人机合作装配作业环境的上下境信息，结合行为语言规则，运用动力系统理论，分析人体复杂行为、事件与场景的理解机制，理解行为并重现行为，揭示人体行为发生、演化机理，最终建立行为描述与理解的新方法。本项目研究对多智能体系统、智能制造系统以及复杂适应系统等相关领域的共性基础理论具有重要意义。

随着智能机器人技术和智能制造的快速发展，智能化的人机协调装配技术是应对复杂机电产品装配系统的高柔性需求的有效途径。针对生产装配的柔性和适应性的需求，项目围绕人机交互合作装配中的人体动作识别、行为分析与智能装配策略展开。设计构建了多目视觉装配智能空间，研究了操作者姿态、运动图像特征的选取、检测和表征方法，获得了手部姿态的描述与人体运动的表征；研究了行为模式的特征与描述方法，建立了基于行为动力学的行为模式；研究了动作和行为识别方法，建立了行为分类与识别模型，获得了较为理想的识别效果；研究了智能装配的检测、策略与控制方法，基于ETH/EIH机器视觉系统的轴、孔位姿测量，提出了人机交互轴孔装配中的空间孔位姿重建方法，建立了基于行为动力学的自主轴孔装配机器人路径规划方法，提出了基于力反馈的宏微机器人轴孔装配策略。项目所取得的研究成果对多智能体系统及智能制造系统等相关领域的共性基础理论具有重要意义，在人机交互、智能家居、视频监控和人机协调装配等领域都有一定应用前景。