自然场景视频人体行为识别特征析取关键问题研究

In recent years, understanding and analysis of human behavior in the video, has become a highly one of the most popular topics in the field of computer vision, image processing and video processing, because of its important scientific research value, a wide range of practical significance and great application prospects. Human action recognition in realistic scenes (unconstrained scenes video, or non-controlled environment) has great challenges due to the tremendous variations that result from camera movement, background clutter, changes in illumination conditions, scale, and viewpoint. For the purpose of simplifying computation complexity and improving algorithm’s performance, the following academic research will be discussed. The approaches of extending multi-parameter affine model, new dynamic camera motion compensation model, as well as foreground object tracking method will be addressed. The approach of spectral embedding is employed for feature clustering, which split up the foreground action into multiple spatio-temporal action parts by embedding the given trajectories into a low-dimension eigenspace through matrix eigen decomposition. To deal with the problem of over-segmentation and under-segmentation resulted from trajectory spectral clustering, a novel density discontinuity detector is proposed for localizing object boundaries according to the density changes. In order to make a greater use of associated information and remove the noise, which is combined with the discontinuous (or unnecessary transition) movement or complex background, a hierarchical structure of features achieved is presented. The mathematical models, frameworks and algorithms to resolve video preprocessing, camera motion compensation, and feature extraction and representation are proposed and developed. Lots of high level academic research papers as well as invention patents will be published. It has significance of the project in both of academic researches and practical applications.

人体行为视频的分析与理解，由于其具有重要的科研价值和广泛社会应用前景，已成为备受关注的研究热点方向。自然场景视频（无约束场景）中人的运动行为识别，受更大程度的由光照、尺度或拍摄视角变化所产生的噪声干扰，加剧了特征析取的难度，影响了识别效果。本项目以简化计算复杂性、提高算法有效性为目标，针对特征析取过程框架、析取算法和表示方法进行深入研究。将拓展多参数仿射模型、设计新的运动补偿和前景跟踪方法；引入谱嵌入思想求解特征聚类，通过矩阵特征分解，将轨迹嵌入到低维特征空间；提出新的密度非连续性检测算子界定对象边界，以应对易出现的过分割和欠分割；以更大程度利用关联信息、并去除由运动不连续（或非必要过渡运动）或包含复杂背景所产生的噪声为目标，构建特征分层结构框架。将建立数学模型、设计计算框架和算法。将发表多篇高水平学术论文、申请发明专利，本项目在视频理解与检索领域具有较大的理论研究价值和实际应用意义。

本课题研究了自然场景人体行为识别的识别，其属于模式识别和计算机视觉等科学领域，本研究中提出的算法和理论有助于本领域相关科学问题的解决。本项目提出了基于改进的显著性轨迹的自然场景人体行为特征析取方法、提出了时空显著性结合显著轨迹的自然场景人体行为识别计算框架和算法、提出了将经典识别方法与深度学习框架结合的自然场景人体行为识别框架和算法。此外，探索了将提出的算法应用到人体目标跟踪等领域，提出了相关的解决技术。.在应用方面，视频人体行为识别研究是计算机视觉领域的前沿方向，同时在运动分析、视频监控和智能交互等领域有着广泛的应用前景。随着众多生物识别技术，如指纹鉴定，虹膜识别，人脸识别，和声纹识别，被相继应用于生产生活之后，视频人体行为识别因其广泛的应用前景，如人机互动，智能监测，和视频搜索，而备受关注。视频人体行为识别技术是图片识别领域在视频方向的延伸，更加注重于时间维度特征的提取与分析。现如今，抖音、腾讯微视等小视频软件家喻户晓，由此产生了大量的视频数据，视频人体行为识别技术能对视频进行理解，分析其中的人体行为，自动标注数据，具有极高的实际应用价值。.对照本课题的立项任务书中的预期目标，我们已经超额圆满的完成了预期任务。在立项任务书中，我们承诺：发表10篇论文，其中SCI等高水平论文5篇，申请发明专利2-3项，培养博硕士人才等；目前，我们已经完成了16篇论文的发表，全部标注了本项目的NSFC项目资助编号，其中SCI收录11篇，其中JCR-SCI一区论文4篇、二区3篇、CCF B会议1篇。本项目申请了自然场景人体行为识别领域专利8项，目前已经授权3项。本项目共培养了接近30名博士和硕士，他们目前已经大部分在高等院校、著名IT企业工作，为国家的进步、人民生活水平的提高努力工作着，这些莘莘学子永远忘不了，在他们最需要支持和鼓励的读研时期，所得的来自NSFC的支持和帮助。