深度学习非凸逼近算法及其在高光谱图像中的应用

Deep learning opens up new research directions in the field of artificial intelligence and machine learning and it is a major breakthrough in science. It is currently an important research topic to reveal the theoretical foundation of the powerful expression ability and the excellent generalization performance, and its implementation for deep learning algorithm. Due to the non-convexity of the cost function for deep learning model, some problems, such as local minima, vanishing gradient, overfitting and the difficulties of parameter configuration, are easy to arise during the algorithms' implementation. These issues are the major challenges for deep learning. This project intends to construct the convex function sequence with parameters to approximate the original non-convex cost function based on the proposed graduated optimization method that is inspired by the convex relaxation theory. Our method provides a new idea to explore non-convex approximation algorithms of deep learning. According to the generalized matrix factorization theory, an appropriate regularization term is constructed to induce a convex regularizer of the network output, and a convex relaxation model for deep learning is established. Simultaneously, deep learning frameworks for hyperspectral images processing are designed based on the proposed non-convex approximation algorithms. This program will provide novel and effective solutions to non-convex optimization problems in deep learning, lay the necessary theoretical support for the implementation of deep learning algorithms and develop a broader space for hyperspectral images processing.

深度学习开启了人工智能与机器学习领域新的研究方向，是科学上的重大突破。深度学习算法的强大函数表达能力和卓越推广性能的理论基础及其算法实现是当前重要的研究课题。由于深度学习模型的代价函数具有非凸性，导致在算法实现过程中易造成局部极小、梯度消失、过拟合及参数配置困难等问题，这些问题是深度学习所面临的重大挑战。本项目拟引入凸松弛思想，提出渐近式优化方法，构造含参数的凸函数序列，用其逼近原始非凸代价函数，探索深度学习非凸逼近算法的新思路。依据矩阵广义分解理论，构造适当的正则化项，诱导出网络输出的凸正则子，建立深度学习凸松弛模型。基于所提出的深度学习非凸逼近算法，设计面向高光谱图像处理的深度学习框架。本项目的研究将为深度学习中非凸优化问题提供新颖且有效的求解方法，为深度学习算法的实现提供必要的理论支撑，为高光谱图像处理开拓出更为广袤的空间。

随着深度学习的深入发展和广泛使用，其损失函数所具有的高度非凸性给传统优化算法带来重大挑战。这些挑战主要体现在优化目标是非凸、高维和NP-难的，这导致在优化过程中易遇到梯度消失，过拟合及泛化性能差等困难。为了克服这些困难，本项目探讨基于优化算法的学习理论，设计适合于自然图像的学习框架和优化算法，以提高深度模型的泛化性能。依据流形学习理论，结合信息论，非凸优化，矩阵优化等工具，对高光谱图像像元去噪、特征提取、分类、混合像元分解等问题进行了深入、系统地研究。此外，针对深度学习中不同任务的特定要求，设计了不同的优化算法和学习框架并对其进行了详尽地分析。具体研究包括：（i）高光谱图像混合噪声移除的秩逼近方法。（ii）用于高光谱图像分类的深度相似网络框架。（iii）基于四元数理论的高光谱图像特征提取方法。（iv）基于流形学习和鲁棒估计的高光谱图像解混算法。（iv）基于概率度量的高光谱图像解混框架。（v）基于稀疏表示和反卷积网络的立体匹配算法。（vi）基于凸松弛思想和泛化理论的非凸优化算法。本项目的研究成果主要以论文形式呈现，目前已发表论文20篇，其中在国际重要学术期刊《 IEEE Transactions on Geoscience and Remote Sensing》、《IEEE Transactions on Image Processing》、《 IEEE Transactions on Cybernetics》等发表论文19篇，在国际会议《International Conference on Wavelet Analysis and Pattern Recognition》发表论文1篇。本项目的研究为高维非凸优化问题的求解提供了新的思路，夯实了非凸优化和图像处理等领域的理论基础，进一步推动了深度学习优化算法的发展。