动态深度与宽度神经网络的泛化误差模型

In the era of big data, deep learning achieved great successes in computer vision area. With the spread of TensorFlow, deep neural networks have become the major method of researches and products in areas of artificial intelligence and computer vision. However, both the huge architecture and large number of nonlinear convolution operations make the deep neural network a black-box. It is difficult to understand the input-output relationship of a deep neural network. The raising of new applications which may yield risk of human life, e.g. autonomous cars, the adaptability to new unseen scenarios and non-stationary environments becomes a key issue of the success of deep learning. In this project, we research the mathematical relationship among data, architecture and parameters of a deep neural network, and its generalization capability. Such that, a localized generalization error model for deep neural network for non-stationary environments will be proposed in this project. The model is expected to provide more in-depth insight to the working mechanism of nonlinear operations of a deep neural network. Based on the proposed error model, we research a localized generalization error model and a training method for deep and broad neural network to provide better accuracies in non-stationary environment. This combines the induction of deep learning and the adaptability of broad learning. Moreover, it is impossible to forces the number of samples in different classes to be balanced in every batch of arriving data, therefore data in non-stationary environments is usually imbalanced. So, this project also researches the influences of imbalanced data to the deep learning and a method to relief these impacts.

大数据时代来临，深度学习在计算机视觉等领域获得了巨大的成功。随着TensorFlow的推广，深度网络已成为人工智能和计算机视觉领域研究和应用产品的主要技术。然而深度网络结构庞大，且有很多卷积等非线性操作，一个深度网络输入和输出之间的关系已成为难以琢磨的黑盒。随着自动驾驶等涉及人身安全的深度学习应用的出现，深度网络面对动态变化及训练中未出现情景的应对能力成为关键问题。本项目研究数据、网络结构参数和泛化能力之间的数学关系，希望获得一个深度网络的局部泛化误差模型以评价其在未见的动态环境中的泛化能力，并试图更深入了解深度网络的非线性运作机理。基于这个模型研究一套结合深度的归纳与宽度的应对变化能力的局部泛化误差模型和训练方法以提升深度网络在动态变化环境中的准确度。另外，动态环境中每批次的数据存在每类样本数不平衡问题，所以本项目亦研究动态样本不平衡对深度学习的影响及应对的方法。

随着大数据时代来临，深度学习在计算机视觉等领域获得了巨大的成功，然而深度网络结构庞大，一个深度网络输入和输出之间的关系已成为难以琢磨的黑盒。而随着自动驾驶等涉及人身安全的深度学习应用的出现，深度网络面对动态变化及训练中未出现情景的应对能力成为关键问题。本项目研究数据、网络结构参数和泛化能力之间的数学关系，推导了深度网络和宽度网络的局部泛化误差模型（LiSSA和BASS）以评价深度网络（自编码器）和宽度网络在未见环境中的泛化能力，然后研究得出一套基于局部泛化误差模型的训练方法，通过最小化模型对输入扰动的敏感度和训练误差以提升深度网络和宽度网络的泛化能力和鲁棒性。其中针对自编码器深度网络的LiSSA进一步被应用到人类动作识别和智能电网上，提升深度网络对这两个问题的泛化能力和鲁棒性。亦针对动态环境中每批次的数据存在每类样本数不平衡问题研究了动态样本不平衡对深度学习的影响及应对的方法。此外，本项目也对动态学习环境中的信息检索问题进行了研究，提出了概念保留哈希等方法。本项目获得的成果已在IEEE Transactions on Neural Networks and Learning Systems、IEEE Transactions on Cybernetics及IEEE Transactions on Multimedia等高水平国际期刊上发表27篇文章，总影响因子达到224.804。团队成员获得2项国家自然科学基金青年项目、中国博士后科学基金及数项省部级项目，纵向项目总经费超过231万元。本项目的研究成果为深度网络和宽度网络的训练提供了新的方法，使其通过最小化泛化误差（模型对输入扰动的敏感度）来优化网络，提升在应用中的泛化能力和鲁棒性，也让我们更加了解深度网络和宽度网络的数据、结构和泛化能力之间的关系。