非参数模型中的低秩和稀疏结构

To overcome the curse of dimensionality, people use the additive structure for non-parametrically modeling the conditional mean and quantile functions. By using sparsity-inducing penalty for variable selection, several authors have developed methods for fitting the additive structure when the number of predictors is very large. However, the finite sample performance of existing methods deteriorates considerably when the number of relevant predictors becomes moderately large. We propose to reduce the number of additive component functions to be estimated using principal components, and generalize this method on generalized additive models and additive quantile regression. To fit the reduced model to the data, we develop a novel algorithm to solve the penalized least loss problem on a rank-constrained manifold with a sparsity-inducing penalty. Our asymptotic theory is trying to show that the resulting estimator has faster convergence rate than estimating without principal component reduction; and this is true even when the reduced model is only an approximation, provided that the approximation error is small. Moreover, we will also prove that the proposed method is able to consistently identify the relevant predictors in the next stage of this proposal. The advantage of the reduced additive models is also illustrated using a simulation study.

在非参数建模当中，研究者经常通过可加结构来避免"维数灾难"。在高维数据下，目前已经有不少工作来提高非参数可加模型的预测效果并识别相关变量。然而，当相关预测变量的数量变得适度大时，现有方法的有限样本性质会显著恶化。本项目提出使用主成分降维的思想来减少待估计的可加分量函数的数量，拟研究一类降维方法用于可加模型，并将其推广到广义可加模型和可加分位数回归模型。结合变量选择的技术，本项目发现这个方法等价于研究非参数模型中的低秩和稀疏并存的结构。为了优化低秩约束下带惩罚项的最小损失函数，本项目通过深入研究增广拉格朗日方法和固定秩流形上的黎曼信赖域方法，拟开发一种新颖的算法用于这类降维模型的未知函数估计和相关预测变量识别。本项目还通过对低秩和稀疏并存结构的研究，完善对所提出估计方法的渐进理论，并用计算机模拟验证这类降维模型的优点与可行性。

为了克服维数诅咒，可加模型作为一种灵活的非参数回归方法而被广泛使用。当预测变量维度非常大时（有时甚至大于样本量），已经有一些方法通过利用诱导稀疏惩罚，来拟合可加模型并实现变量选择。然而，尽管这些方法有良好的渐进特性，但它们假设相关预测变量的数量相对有限。然而，当相关预测变量的数量变得适度大时，这些方法的有限样本性能可能会大大恶化。本项目提出了一种新的方法，通过学习由可加分量函数共享的预测子空间，来减少需要非参数化估计的未知函数的数量。结合子空间学习与诱导稀疏的惩罚性最小二乘法，本项目又开发了一种高效的算法，实现矩阵流形优化和矩阵上的近似阈值运算的计算。本项目还研究了该算法的理论收敛特性。所提出的方法在模拟研究和实际数据例子中均显示出与现有方法相比的优势。