基于三通道判别卷积神经网络和迁移学习的红外人体行为识别

Infrared human action recognition has attracted increasing attention for it has many advantages over visible light, that is, being robust to illumination change, and can work 24/7. However, it faces several key problems: 1) comparing to visible light, the texture of infrared image/video is much weaker, so relevant feature should be designed to solve this problem; 2) the fact of weak texture property of infrared image/video will increase the diversity of within-class and similarity of between-class of infrared human action, which makes the recognition of infrared human action more difficult; 3) the infrared equipment is expensive, which leads to the lack of infrared data, and makes the classification model cannot be well-trained. Aiming at the aforementioned problems and based on the applicant’s previous works (these works have been published on TNNLS, TIP, Complexity, Neurocomputing, etc.), this project will built three-streams CNN and discriminative CNN to extract feature for infrared human action, and then employ transfer learning to reduce the gap between human action from visible light and infrared imaging, so the data from visible light can be used to expand the training set of infrared human action. In this way, the aforementioned problems can be solved successful, and the state-of-the-art performance of infrared human action recognition can be promoted.

基于红外的人体行为识别由于不受光线条件影响，可全天候工作等优点，有着巨大的应用价值。然而，现阶段红外人体行为识别面临以下几个亟待解决的问题：1）红外图像/视频纹理信息较弱，需要设计针对性的特征提取方法；2）人体行为存在类内离散性和类间相似性的问题，而红外数据纹理较弱的特点会进一步突出该问题，这将增加红外人体行为识别的难度；3）红外设备相对昂贵，导致红外数据较少，无法充分训练分类模型，也是一个需要解决的问题。针对上述问题，结合申请人之前的研究工作（研究成果发表在TNNLS、TIP、Complexity、Neurocomputing等领域主流期刊上），本项目拟构建三通道CNN和判别式CNN为红外人体行为提取特征，使用迁移学习减小可见光和红外数据间的模态差异，利用可见光人体行为数据扩充训练样本，从而解决上述几个问题，最终提高红外人体行为识别的技术水平。

本项目围绕红外图像/视频中人体纹理特征少的特点，提出了有效的特征提取方法；设计了针对性的迁移学习方法，解决了红外图像/视频样本少的问题；构建并验证了基于三通道卷积神经网络和迁移学习的红外人体行为识别系统。取得的代表性工作包括：1）构建了三通道卷积神经网络结构提取鲁棒的红外人体行为特征；2）提出了基于特征对齐与归纳的可见光—红外跨模态迁移学习框架；3）提出一种基于深度适配融合网络的图像到视频跨媒体行为识别算法。本项目产生了一批高水平成果，包括在IEEE Transactions on Image Processing等顶级期刊发表论文；申请国家发明专利3项，其中1项已经获得授权；该项目的部分研究成果已应用于智慧操场，并与企业达成初步合作意向。