面向服务机器人的移动视角行为理解和人机交互多模算法研究

Fast development of artificial intelligence and aging problem create a large demand for service robots in our society. For that, this project studies on activity understanding, prediction of users and human-robot collaborations to solve problems of perception and interaction in human robot interactions (HRI). Our research contains: (1) The research constructs a moving-view human activity dataset in HRI. Since feature spaces of moving-view activities are not in common, the research proposes a 3D Moving-View Transfer Learning approach to solve it for activity understanding. (2) Considering that human activities are complexly composed of several action categories, and activity understanding is required to be in real time, the research proposes the slide window LSTM and the hierarchical sematic model for a real-time and hierarchical understanding of user activities. (3) The research collects multimode information of user activities using perception algorithms or in manual mode, and proposes a model for semantic projection of multimode features to realize intelligent understanding. (4)To realize human-robot collaboration, the research modifies the reinforce learning to model state transfer of user activities for user intention prediction. Moreover, the research learns interaction experience model from human-human interactions, and promotes human robot interactions. In a service robot centroid mode, the research realizes activity recognition and prediction, developing a HRI system. It devotes to realize a natural and influence interaction between human and service robots. It supports the research on intelligent technology for robots.

人工智能快速发展和社会老龄化趋势增强了社会应用对服务机器人的需求，本项目对服务机器人与人进行人机交互存在的认知、交互等问题展开研究。本项目研究内容包括（1）建立HRI移动视角人体行为活动数据库，针对移动视角人体活动数据的特征空间不重合问题，提出3D 移动视角迁移学习模型（3DMVTL）实现用户行为理解；（2）根据HRI人体活动的复杂组合和实时理解需求，提出滑动窗LSTM算法和语义分层模型实现分层实时理解；（3）自动学习和网络征集等多渠道获取人体行为活动的多模态信息，提出多模特征语义映射方法进行类人智能化理解；（4）改进增强学习算法模拟人体活动中状态转移，建立行为序列推演模型，预测用户动作趋势，学习人与人动作交互经验模型，驱动机器人响应用户动作。本项目研究以机器人为中心，理解、预测用户行为活动，建立服务机器人与人之间的人机协作系统，实现自然顺畅的人机交互，是国内机器人智能技术开发的有益补充。

本项目基本根据原项目计划展开研究，超额完成原规划研究目标。针对HRI应用场景中移动视角行为识别问题缺乏数据支撑算法验证问题，收集用于任意视角动作分析的UESTC RGB-D action dataset数据库，包括RGB视频、深度和骨架序列。该数据库公开在GitHub网站，受到国内外同行关注。为了更好地把握当前研究现状，对用于 HRI 系统的行为活动理解相关论文和系统进行调研，撰写综述，论文已发表。本项目将更多注意力放在研究新的智能学习算法实现移动视角行为活动理解，提出了Attention Transfer (ANT)、Coarse-to-fine、小样本学习等算法，解决移动视角中的特征表达难题。在算法研究的基础上，拓展算法在人与机器人交互系统中的应用，建立人机交互系统。本项目超额完成预期研究成果中的论文发表、专利申请，以及学生培养等任务。