基于逼近的Boosting算法及其在回归中的应用

The Boosting algorithm, as an ensemble learning method to improve accuracy, has made remarkable achievements in the area of pattern recognition over the past decade. The basic idea of Boosting refers to the general process of producing a very accurate prediction algorithm, which we call "strong" learners, by appropriately combining rough and moderately inaccurate ones, or, "weak" learners. Compared with the prosperous applications in pattern recognition, Boosting owns relatively less results in both theories and applications regarding nonparametric regression. This project, from the viewpoint of constructive approximation, generalizes Boosting from classification to some regression issues. The approximation function is constructed from general or special function spaces based on the "weak" learners therein and, the regression function is approached by it. According to such an idea, we would accomplish the following objectives...1. Combining the Boosting algorithm, we will design the specific constructive steps and determine the fitting and iteration steps for general function spaces;..2. Utilizing the techniques of parametric smoothing, the number of iterations under different circumstances will be fixed;..3. The constructive method proposed in goal 1 will be applied to some kinds of new function spaces. Such as the space of maxima of minima functions which includes the perceptrons as its special case, neural networks, etc. We would also prove that the rate is optimal considering the convergence of such approximated functions to regression functions...The research on the effects of Boosting algorithm in nonparametric regression owns its double significance both in theory and reality.

Boosting 集成学习算法是一类可提高任意给定学习算法准确度的方法，过去的10 多年来在模式识别领域已取得卓越成效。其基本思想就是通过产生若干简单的、精度比随机猜测略好的粗糙估计集成构造出一个高精度估计。相比在模式识别(分类问题)中的广泛应用，这类算法在非参数学习的回归问题中无论是理论还是应用都相对较少。本项目从逼近的角度，将Boosting算法从分类问题推广到一些回归问题，利用该算法构造基于一般及特殊函数空间“弱学习器”的逼近函数，并实现对回归函数的近似。本项目主要研究 1.结合Boosting 等算法设计具体构造步骤，确定适用于一般函数空间的拟合、迭代方式，2.利用参数光滑化技巧讨论不同情况下迭代次数的确定，3.将构造方式用于感知器、神经网络等在内的几种不同应用类型，对其进行误差分析并研究最优阶成立条件。项目对于Boosting算法在非参数回归问题中的研究具有理论与实际的双重意义。

Boosting 集成学习算法是一类可提高任意给定学习算法准确度的方法，过去的10 多年来 在模式识别领域已取得卓越成效。本项目展开了对各类基函数的研究，涵盖三角级数、有理函数、有理双线性函数、核函数等；以上述基函数作为弱学习器，利用Boosting算法等集成方法构造了基于各类基函数的逼近函数，并设计具体算法步骤，包括拟合迭代方式的选择、迭代次数的确定、光滑参数的选取等；进而讨论并研究构造函数对回归函数的逼近性质与逼近度等问题。项目对于Boosting算法在非参数回归等问题中的研究具有理论与实际的双重意义。