基于多模态视觉模型的行为事件分析研究与应用

The low illumination at night, and the low visibility of the heavy haze environment have brought huge challenges to the traditional video surveillance, while multimodal vision technology is an effective means to solve these problems. The frontal imaging part of the multimodal vision model employs the application popularization and color reducibility of visible videos, combined with the illumination independence and strong particle penetrability of infrared thermal videos; the back-end analysis utilizes the complementary characteristic of the multimodal video information, and then finally achieves the goal of intelligent video analysis in the harsh environment. On the basis of building the multimodal vision model, this project will focus on the scientific problems including object detection, object characteristic representation and recognition, multimodal object tracking, individual behavior recognition and group event analysis in the multimodal condition. The main tasks are to propose the sparse representation based background modeling algorithm and object tracking algorithm for the sparseness of multimodal data; to design the graphic representation based behavior recognition algorithm and event analysis algorithm on account of the relationship between the behavioral events; to constitute the complete algorithm framework with bottom-level processing, middle-level analysis and high-level semantic parsing for multimodal video data. The study of this project will powerfully boost the formation of the theoretical system and the application promotion of the all-time and all-weather intelligent video analysis.

传统基于可见光的视频监控系统在夜晚低照度场景及雾霾低可见度环境下性能急剧下降，多模态视觉技术是解决这一问题的有效手段。多模态视觉模型在前端成像利用可见光视频的应用普及性和颜色还原性，结合红外熱像视频的照度非依赖性和对微粒的强穿透性，在后端分析利用多种模态视频数据信息互补的特点，实现恶劣环境下视频智能分析的目的。本项目拟在构建多模态视觉模型的基础上，研究多模态条件下目标检测与目标特征表达识别、多模态目标跟踪、个体动作行为识别与群体事件分析等科学问题，针对多模态数据稀疏特性提出基于稀疏表示模型的背景建模算法和目标跟踪算法；针对行为事件的关系特性提出基于图表示模型的行为识别算法和事件分析算法，形成多模态视频数据下由底层处理、中层分析到高层语义解析构成完备算法框架。本项目的研究将会有力的推进全天时全天候智能视频分析的理论体系形成和系统应用推广。

传统基于可见光的视频监控系统在夜晚低照度场景及雾霾低可见度环境下性能急剧下降，多模态视觉技术是解决这一问题的有效手段。多模态视觉模型在前端成像利用.可见光视频的应用普及性和颜色还原性，结合红外熱像视频的照度非依赖性和对微粒的强穿透性，在后端分析利用多种模态视频数据信息互补的特点，实现恶劣环境下视频智能分析的目的。本项目在构建多模态视觉模型的基础上，研究多模态条件下目标检测与目标特征表达识别、多模态目标跟踪、个体动作行为识别与群体事件分析等科学问题，形成多模态视频数据下由底层处理、中层分析到高层语义解析构成完备算法框架。本项目的研究将会有力的推进全天时全天候智能视频分析的理论体系形成和系统应用推广。..在项目运作期间，发表高水平期刊论文 32 篇，会议论文20 篇；申请专利9 项，并形成多模态智能视频分析原型系统，申报软件著作权 1 项；建立规范化多模态视觉实验数据集，推动多模态视觉分析研究的发展；在视频分析应用方面，基于项目研究的算法进行车辆检测、跟踪和识别研究，开发了一个高速卡口车辆检测识别系统；举办学术研讨会10余次；参加国际会议 10 次，国内会议 10 次；培养4位博士研究生和22位硕士研究生。