高阶高维张量回归的优化理论与算法

Tensor-based modeling and computation emerge prominently with the rapid development of modern science and technology, and demands from practical applications in the big data era. The concept of high order high dimensional tensor regression（HOHDTR） is exactly a high order tensor-based statistical regression in the high dimensional setting, in which the unknown coefficients admit a high order tensor structure. The HOHDTR has a wide range of applications in many areas such as medical imaging and diagnosis, machine learning and artificial intelligence, data mining, spatio-temporal forecasting ,and social network. The involved core mathematical model can be formulated as a constrained high order tensor optimization problem, which is generally NP-hard. This project aims to study the theory and algorithms for HOHDTR problems, which mainly contains the following three parts: (1)Explore the properties in statistics and in optimization for the HOHDTR with constraints generated from some intrinsic sparsity, low-rankness, and other prior information on structures of the tensor in question, and its possible regularization and relaxation tensor optimization models, in order to build up the corresponding relaxation theory and solution analysis; (2) Propose highly efficient and robust algorithms and establish their convergence analyses for large-scaled tensor optimization problems from HOHDTR problems; (3) Conduct numerical experiments on synthetic data and data from application problems in data mining and machine learning to provide an efficient software package for handling tensor optimization problems in practice. This project is of significant importance in scientific research and in real-world applications, not only for providing new theory and methodology for tensor regression analysis, but also for promoting the cross and integration of statistics, tensor analysis and optimization, and hence for triggering the rapid development of the statistical optimization which is a brand new interdisciplinary field.

随着现代科技的飞速发展以及大数据时代的实际应用需求，基于张量的建模与计算应运而生且备受关注。高阶高维张量回归（HOHDTR）是基于高阶张量的高维统计回归，被广泛应用于医学成像疾病诊疗、机器学习与人工智能、数据挖掘、时空预测、社交网络等领域。其核心数学模型为约束高阶张量优化，属于NP-难问题的范畴。本项目欲开展HOHDTR的优化理论与算法研究，主要内容包括:在高维情况下考虑带有稀疏低秩以及先验结构的非凸约束HOHDTR及各种正则化松弛优化模型的统计性质与优化理论；设计高效稳定、适合求解大规模张量优化问题的算法并建立收敛性分析；进行数值实验及在数据挖掘、机器学习等领域的应用研究,编写实用软件包。本项目的研究不仅为张量回归分析提供了新理论与新方法，而且能够促进统计学、张量分析与最优化的交叉与融合，推动统计优化这一新兴交叉学科的快速发展，具有重要学术研究意义和实际应用价值。

高阶高维张量回归是基于高阶张量的高维统计回归，被广泛应用于医学成像与疾病诊疗、机器学习与人工智能、数据挖掘等众多领域。如何合理构建高阶高维张量回归问题的稀疏低秩约束优化模型与正则化模型，并设计高效稳定的优化求解算法进行相应的参数估计，克服与规避高阶高维张量结构带来的超高计算复杂性及计算存储瓶颈问题，是张量回归的优化理论与算法研究的重点与难点,这些研究内容为统计回归带来了新的挑战和机遇，也将进一步促进稀疏优化与大规模统计计算的发展，因而具有重要的理论研究意义。本项目围绕高阶高维张量统计回归的理论、算法与应用等方面展开研究，在张量回归模型统计理论、稀疏回归的优化理论与快速算法、应用研究三个方面取得了较好的研究成果。主要包括：(1)针对L2,0-范数组稀疏约束与组稀疏正则的广义线性回归模型，建立特征选择一致性与最佳系数估计等统计理论；针对低秩张量Huber回归问题，建立统计误差界理论等；(2)针对高阶高维张量回归中凝练出来的稀疏优化问题，建立相应的高效稳定二阶算法，包括求解非线性稀疏优化本原模型的Lagrange-Newton算法，以及求解稀疏分组SLOPE正则回归优化的半光滑牛顿增广Lagrange算法等；(3)将理论与算法应用于地铁节能时刻表、3D人脸表情识别、因特网流量异常检测等三类重要实际问题，数值实验结果验证了方法的有效性与实用性。如上研究成果不仅丰富了稀疏优化的理论与算法，而且有助于统计、优化、机器学习、张量分析的交叉融合。通过该项目的开展，我们将理论探索、算法设计、应用实践有机融合，兼具理论研究意义与实际应用价值。