基于元姿态的无约束环境中三维人体姿态估计研究

Since there is no assumption on human pose, background and other factors, 3D human pose estimation in unconstrained environments is a key technology in the field of intelligent video surveillance and active human-robot collaboration. This project proposes to explore the 3D human pose estimation in unconstrained environments based on meta human pose set, which aims at solving the contradiction between the high diversity of human poses and the dependence of existing algorithms on complete datasets. To capture the relationship among human poses, this project designs a new representation model based on graph neural network. By jointly learning the group discovery and vertex embedding of this graph model, a meta human pose set is extracted. By modeling the relationship between human pose and image, this project puts forward a meta training image set mining method guided by meta human pose set. To solve the hybrid supervision learning problem, this project studies the 3D human body pose estimation model based on structural consistency. By studying the extraction and utilization of meta human pose set, the project constructs a representative sample set conforming to the human pose distribution in unconstrained environment, which is then used to train a model for 3D human pose estimation. Through this research, the project improves the performance of 3D human pose estimation in unconstrained environments, which provides technical support for intelligent video surveillance and human robot collaboration.

由于对人体姿态、背景等因素均无限制，无约束环境中三维人体姿态估计是智能视频监控、人-机器人主动协作等领域的关键技术。针对无约束环境中人体姿态的高度多样性和现有算法性能依赖完备数据集的问题，本项目提出基于元姿态的无约束环境中三维人体姿态估计研究。本项目拟研究基于图网络的人体姿态关系表征模型，提出基于群体发现的元人体姿态集提取方法；研究人体姿态与图像内容的关联建模，提出基于元姿态指导的核心训练图像集挖掘方法；研究混合监督的模型训练方法，提出基于结构一致性动态先验的三维人体姿态估计模型。通过研究元人体姿态集的提取与高效学习，构建符合无约束环境下人体姿态分布的代表性样本集合，突破无约束环境中三维人体姿态估计技术，为智能监控和人-机器人协作等应用提供技术支持。

由于对人体姿态、背景等因素均无限制，无约束环境中的三维人体姿态估计在智能视频监控、人-机器人主动交互等多个领域具有重要的应用价值。围绕姿态多样性高和数据依赖强的问题，本项目开展了基于元姿态的无约束环境中三维人体姿态估计研究。（1）针对人体结构建模与姿态增强，设计了一种基于门控图网络的人体表征模型，使用图网络建模人体关节点的依赖关系，提高了人体姿态的表征效率；提出了一种人体姿态增强方法，从旋转方向、身体长度和位置等三个方面对输入姿态进行增强，提高姿态分布范围。（2）针对核心数据集构建问题，提出了一种自适应的视角转换模块，缓解了视角对姿态的影响；提出了一种基于长时依赖的视频质量预测与关键数据提取方法，突破了视频数据价值评估与核心数据提取问题；设计了一种基于局部增强的语义引导视频片段提取方法，提高了数据提取的准确性。（3）针对三维姿态估计中遮挡等问题，提出了一种基于图模型的遮挡人体姿态估计方法，使用关节点依赖关系推理遮挡节点的位置；提出了一种基于对比相关的跨模态行人再识别方法，动态挖掘模态之间的关键特征，提高了行人再识别的效果。通过三年的研究，突破了开放环境中人体姿态估计关键技术，相关成果在IEEE TIP、PR等高水平期刊和会议上发表论文6篇，申请专利2项，其中授权1项，参加国内外高水平会议5次，联合培养博硕士研究生6名，完成了申请书中的研究目标。