基于结构化学习理论实现手势预判的方法研究

As a non-contact way of interaction, vision-based gesture recognition facilitates human-robot interaction with more easiness, naturalness and convenience without wearing extra devices. It can be applied from games control to vision-enabled robot control, from virtual reality to smart home systems. Although current gesture recognition can achieve high accuracy rates, existing gesture detectors are trained to recognize complete gestures only and the entire gesture is required for a reliable decision. Early detection of gestures, one of the main innovations in this project, is a relatively unexplored problem. Traditional learning methods did not exploit the structured information between sample data. In this proposal, we introduce structural learning algorithms to distinguish meaningful gestures from unintentional hand movements. The proposed method is efficient due to the structured information in the three-dimensional trajectories of depth images.. Autonomous navigation of a service robot highly depends on the modeling of environment and the accuracy of positioning; however, problems exist to be solved in the environment full of relatively irregular-shaped obstacles and unexpected activities. Without maps or prior information of obstructions, robot hardware platforms are generally more expensive and prone to collisions, resulting in unnecessary loss of damage. In this study, a depth-perceptive gesture early detection system is proposed to combine with the robot navigation in dynamic environment. Hand postures are estimated in the three-dimensional space and the real-time trajectories are analyzed in depth images. Furthermore, structured information is provided as characteristics of gesture in the optimization algorithm for early detection, and the descriptive ability of gesture is enhanced to broaden the scope of dynamic applications. A gesture early detection system is designed to provide control strategies for intelligent navigation in dynamic environment and a new way to interact with service robots. Gesture-guided robots are capable of real-time access to the environmental information for autonomous and semi-autonomous coordination in the navigation system.

当前的手势识别技术虽可达到较高的识别率，然而现有的识别方案只着重于识别完整的手势，需要在手势动作完全结束后，才能进行识别及判断，如何在手势动作未完全结束前即侦测并鉴别出手势，是一个相对较新的问题，本项目的主要创新点即是手势预判功能的实现。有鉴于传统的学习方法在处理分类时没有利用到样本数据间的结构关联信息，本项目提出基于结构化学习的方案，利用深度图像得到三维手势轨迹样本，因而得到更高的效率。此外，对操作者有意识的手势与无意识的手部动作做出区分，获得更自然的交互体验。. 服务机器人的自主导航仍存在一些亟待解决的问题，包括在没有地图或障碍物先验信息的情况下，容易出现碰撞形成伤害，基于深度感知的手势预判结合机器人导航的研究，为服务机器人在动态环境中的智能导航提供新的交互方式和控制策略，机器人在动态环境中结合动态手势系统实时获取周围的环境信息，实现自主和半自主相互协调的导航系统。

随着计算机视觉技术的蓬勃发展，手势识别在人机交互中的方便和非语言交流中起着越来越重要的作用。然而，当在数据库中考虑足够有意义的手势时，由类似手势引起的混淆会带来固有的错误。在本项目执行过程中，提出了一种类似手势识别的自动特征提取，用于在类似手势中区别混淆。除了方向特征外，还提取了四个额外的创新功能，以在包含10个数字和26个字母的实验数据库中显着区分所有相似的手势。与传统方法相比，提取了几个相似的手势作为特定特征，所提出的方法将自动特征提取相似的手势区分开来。实验结果表明，该方法具有较高的识别率和多功能性。该项目取得的学术成果包括 2 篇 SCI 论文和 6 篇 EI 论文，培养硕士生 3 名，申请发明专利 2 项，完成系统样机 1 套。