基于跨域深度学习的复杂视频场景分类方法研究

Deep learning, which can compose low-level features to form higher-level features, is a promising learning method about multi-layer network. When deep learning model is applied for classifying complex video scene, a large number of video scene samples are indispensable for training its parameters because there are a lot of semantic objects and tremendous variations from camera motion and background in the videos. Since manual labeling of video scenes is laborious and time-consuming, we can only get a small number of complex video samples which are not enough for us to train an effective video scene classification model with good classification performance and generalization capability.So we need a great number of auxiliary samples. However, after we get these samples from Internet, deep learning model will face other problems: how to effectively extract high-level semantic features and how to build video scene classification model by using the cross-domain samples. To solve these problems, we will study Complex Video Scene Classification Method Based on Deep Learning with Cross-Domain Sources in this project. And we will focus on exploring the following four key technologies:(1)the method for automatically constructing auxiliary scene dataset from Internet sources with many irrelevant samples,(2) the high-level feature extraction and representati- on method based on deep learning with cross-domain samples, (3) the effectively learning and computation method of deep learning model based on cross-domain sample sets，and (4) the method for building complex video scene classification model by fusing high-level classification information of cross-domain sources. The final scene classification model can preserve good behavior classification performance and generalization capability. When this project is finished, the theories or methods for building complex video scene classification model based on deep learning with cross-domain sources will be systematically formed. Meanwhile, an antitype system for complex video scene classification will be built and elementarily applied in the video scene analysis fields of visual surveillance and digital television.

深度学习能够通过组合底层特征形成更加抽象的上层特征，是一种极具前途的多层网络学习算法。当应用于复杂视频场景分类建模时，由于场景的多语义性和拍摄条件的多变性，它需要大量样本以学习深度模型的参数，但是,在复杂视频应用领域往往只能有效获取少量标记样本，需要借助网络辅助资源，此时，深度学习又面临跨域特征提取和跨域建模的问题。因此，本项目拟开展基于跨域深度学习的复杂视频场景分类方法研究,重点研究四个方面的内容：（1）大规模网络辅助资源样本集的自动构造方法；（2）基于跨域资源深度学习的场景高层语义特征提取与表示理论；（3）基于跨域样本集的深度模型参数有效学习策略和快速计算方法；（4）基于上层分类信息融合的复杂视频场景分类器跨域建模方法。通过本项目的实施，将在理论上形成系统的基于跨域资源深度学习的复杂视频场景分类理论和方法，同时形成一个原型系统，在视频监控和数字电视视频场景分析领域得到初步应用。

由于具有广泛的应用价值，场景分析与识别是计算机视觉领域的一个研究热点，很多国家、组织投入了大量的人力和物力，对场景内容分析与识别进行了大量研究。2006年，Hinton等人提出了一种深度学习方法。该方法具有多层非线性映射的深度结构，可以完成复杂函数逼近，能够通过多层抽象，最终提取有效的高层语义特征。但是，由于场景的多语义性和拍摄条件的多变性，为了有效训练深度模型参数，需要大量样本，但是在很多应用领域往往只能获取少量标记样本，需要借助网络辅助资源，此时，深度学习又面临跨域特征提取和跨域建模的问题。因此本项目围绕网络样本集构造、基于深度学习的场景高层特征提取与表示和场景分类器跨域建模三个方面的问题展开研究，取得了以下成果：研究并实现了一种基于迭代学习聚类的资源样本集构造方法，能够避免大量的人工标注，自动构造大规模数据集；通过深入研究和融合深度残差网络模型、卷积夸张、LSTM深度网络模型、基于SAR变换的数据增强和多尺度融合预测理论和方法，提出了一种基于SegResNet深度网络的场景图像理解方法，该方法具有较好的图像场景理解性能；本项目研究提出了一种基于预训练监督项的深度神经网络分类方法，该方法在针对目标小样本集的迁移学习过程中，能够明显提高网络模型的收敛速度，提高识别性能，避免欠学习问题的发生，该方法特别适用于小样本集的嵌入式分类应用。此外，本项目在充分理解Jeffrey Dean等人提出的基于Sandblaster L-BFGS的深度模型快速计算框架的基础上，实现了一种基于分层控制的GPU深度模型参数快速计算方法，搭建了适用于大规模场景分类的软件系统。