基于信息理论学习的单类分类关键问题研究

One-class classification extensively exists in the practical application fields of intrusion detection, fault diagnosis, etc. It can efficiently deal with the problem of training classifiers with samples in only one-class and the problem of extreme class imbalance. The existing one-class classification methods have the following shortcomings. They are unfit for handling the high-dimensional data. Their classification boundaries are not compact enough. Their optimal parameters cannot be obtained. The complexities of training and testing these approaches are high. The targets of this project are overcoming the above-mentioned shortcomings and performing application research on practical examples. The problems to be solved contain: 1. Establish efficient feature selection strategies for one-class classification. Design models of feature-weighted one-class classifier. Propose information theoretic learning based feature extraction methods; 2. Propose Renyi entropy and correntropy based selective ensembles of one-class classifiers. Study relationship between generalization abilities and fuzziness of base classifiers in ensemble of one-class classifiers; 3. Establish the parameter selection strategies based on the Bayesian framework and the intelligent optimization techniques for one-class classifiers; 4. Establish entropy based sample selection method with active learning for one-class classification. Propose the method of training support vector data description (SVDD) in primal space. Establish region segmentation based SVDD; 5. Apply the proposed methods to image retrieval, intrusion detection, and fault diagnosis. Furthermore, propose one-class classifiers fit for handling the problem of extreme class imbalance.

单类分类广泛存在于入侵检测、故障诊断等实际应用领域中，它能有效解决仅有一类样本用于训练分类器的问题和类别极端不平衡问题。现有单类分类方法存在以下缺陷：不适于处理高维数据；分类边界不够紧致；无法选取到最优参数；训练和测试复杂度过大。本课题旨在克服现有方法的上述缺陷并开展实例应用研究，拟解决的问题是：1.为单类分类提出高效的特征选择方法，建立特征加权单类分类器模型，提出基于信息理论学习的特征提取方法；2.提出基于Renyi熵及相关熵的单类分类器选择性集成，研究单类分类器集成的泛化能力与基分类器模糊性之间的关系；3.为单类分类器建立基于贝叶斯框架及智能优化技术的参数选取策略；4.为单类分类提出基于熵的主动学习样例选取方法，提出在原空间中训练支持向量数据描述(SVDD)的方法和基于区域分割的SVDD；5.将上述模型方法用于图像检索、入侵检测及故障诊断，并提出适于处理类别极端不平衡问题的单类分类器。

单类分类是目前机器学习领域中的一项重要研究内容。一些单类分类算法已在异常检测、目标识别、图像分类等各种应用问题中取得了较好的效果。然而，不同于传统的二分类和多类分类问题，单类分类问题中仅有一类样本可用作训练样本，因此传统的二分类或多类分类算法无法直接用于解决单类分类问题。本项目针对单类分类中存在的关键问题开展了五方面的研究：1. 对于单类分类的维数约减，提出了基于相关熵和Log型惩罚函数的鲁棒稀疏编码，并提出了基于深度PCA子空间极限学习机方法；2. 针对单类分类器集成，提出了基于鲁棒AdaBoost的单类支持向量机集成，并提出了基于旋转的单类支持向量机集成；3. 在对单类分类器参数选取的研究中，提出了实数编码的量子共生算法，并提出了一种基于改进的初始权重设置方式的加权极速学习机；4. 在对快速单类分类器的研究中，提出了基于可缩放hinge损失函数的鲁棒单类支持向量机，提出了基于C损失函数的鲁棒单类最小二乘支持向量机，并提出了基于共轭梯度算法的极速学习机；5. 在应用研究中，对网络入侵检测、图像处理和推荐算法开展了应用研究，分别建立了相应的机器学习模型。本项目研究成果在机械故障诊断、网络入侵检测等领域中具有广泛的应用前景。