神经网络逼近与深度学习理论

Deep neural networks or deep learning techniques, which simulate the mechanisms of human brain, have made significant progress and have shown outstanding performance in many applications. Deep learning is honored as a breakthrough in artificial intelligence and machine learning. Nevertheless, its mathematical theory is still lacking. It is very important and urgent to understand the powerful ability of representing functions, and to reveal the excellent performance and the theoretical foundation for deep learning algorithms. This project aims at developing aspects of nonlinear approximation and learning for deep neural networks from the perspective of approximation theory of functions. We will explore the nonlinear approximation properties and learning performance for deep neural networks approximating to composite functions. We try to establish a theoretical framework of error estimation for deep neural networks. We will also analyze and compare the difference between shallow and deep neural networks and establish the relation between the approximation ability of neural networks and the number of layers. This will help us to find the feature of deep learning and to guide the design of new deep learning methods. Based on the theoretical results we will try to design some novel deep learning algorithms for different kinds of applications, such as pattern recognition and hyperspectral image classification.

模拟人脑机制的深度神经网络或深度学习技术发展迅猛，在许多领域有广泛的应用，取得重大进展，被誉为人工智能和机器学习领域的重大突破。然而，深度学习算法的数学理论基础研究仍十分欠缺。理解深度学习算法所具有的强大函数表达能力，揭示算法所表现的卓越性能，研究算法的数学基础显得尤为重要和迫切。本项目从函数逼近论的角度研究深度神经网络非线性逼近问题和深度学习的理论基础。分析深度神经网络算法的特性，探索其逼近性质与其函数表达能力的联系，研究深度神经网络算法对复合函数的逼近和学习能力；讨论深度神经网络算法的收敛性，建立算法误差估计框架；分析和比较深度神经网络与浅层神经网络的函数逼近性质差异，建立神经网络逼近能力与网络层数的联系，揭示深度学习算法优良性能的本质属性，并为设计新的深度学习算法提供指导。在理论研究基础上，设计深度学习算法，探索算法在模式识别和高光谱图像分类等问题中的应用。

本项目从逼近论视角研究深度神经网络的非线性逼近和深度学习的理论基础。通过分析目标函数的分层复合结构，探究深度神经网络算法的基本逼近性质及其与表示学习的关系。涉及数学、统计学和计算机科学等多学科的交叉融合，是富有挑战的研究课题之一。项目组成员认真履行申请书的承诺，积极工作，取得了一些有意义的研究结果，较圆满完成项目研究计划。为提升学习效率，建立了一种适用于高光谱遥感影像分类的轻量型深度学习网络框架；将knockoff推断技术与深度神经网络相结合，构建了一种新的特征选择算法；研究了基于广义协同表示学习的高维数据分类方法的理论、算法及其在人脸识别中的应用。将理论研究成果应用于图像分类和生物神经元重构等实际问题，得到一些有意义的应用研究结果。