高维流式大数据的增量特征提取算法研究

Big data streaming is a process in which large streams of real-time data are processed with the sole aim of extracting insights and useful trends out of it. It is an important research domain in big data era and has wide application prospects in many fields, such as business intelligence, marketing and public services. However, big data stream has exhibited a remarkably dynamic characteristics and its noise distribution is also more complicated, at the same time, there are unavoidable errors in feature extraction for the high dimensional big data stream. Such situations present new challenges for the traditional machine learning and data mining algorithms, especially in the big data stream feature extraction, which is important for the data stream mining. This project thus proposes to optimize the incremental feature extraction algorithms for big data steam. Firstly, according to the dynamic changing characteristics of big data streams, an adaptive incremental feature extraction algorithm is proposed to realize the effective learning of high dimensional data stream. Secondly, a robust incremental feature extraction algorithm is proposed to achieve robust learning of high dimensional data stream according to the big data streams in the environment with noise, this algorithm can also reduce the computational complexity. Finally, an incremental feature extraction optimization algorithm based on regularization is designed. This is aiming at the unavoidable dimensionality reduction errors generated in the feature extraction process for the high dimensional data stream. In summary, this project enacts an effective theory support and technology for adaptive incremental learning in big data mining by the optimization of feature extraction for the high dimensional big data stream.

流式大数据作为大数据的一种重要形态，有着广泛的应用前景。然而，在高维流式大数据的处理过程中，现有机器学习和数据挖掘算法存在难以处理动态特征、噪音环境下算法复杂度高以及特征提取中误差大等问题。为了发现高维数据流内在的结构信息，为后续学习任务的高效顺利进行提供支撑，本项目针对高维流式大数据的增量特征提取算法展开研究：首先，针对流式大数据特征呈动态变化的特点，研究自适应增量特征提取算法，实现高维数据流的有效学习；其次，针对噪音环境下的流式大数据，研究具有鲁棒性的增量特征提取算法，实现高维数据流的鲁棒学习；最后，针对高维数据流特征提取过程中不可避免地产生的降维误差问题，研究基于正则化的增量特征提取优化算法，得到适应不同环境的增量学习优化模型，提高学习效率。项目研究成果将为流式大数据的增量学习提供有效的理论支撑和良好的技术借鉴，实现高维数据流的高效学习，具有重要的理论意义和应用价值。

流式大数据作为大数据的一种重要形态，有着广泛的应用前景。为了发现高维数据流内在的结构信息，为后续学习任务的高效顺利进行提供支撑，本项目针对高维流式大数据的增量特征提取算法展开研究：首先，针对流式大数据特征呈动态变化的特点，研究了自适应学习因子，提出一种新的基于增量切空间的自适应流式大数据学习算法，实现高维数据流的有效学习；其次，针对噪音环境下的流式大数据，设计鲁棒增量特征提取算法，提出了一种基于鲁棒回归的多标签分布降维算法，实现高维数据流的鲁棒学习；最后，针对高维数据流特征提取过程中不可避免地产生的降维误差问题，研究基于正则化的增量学习优化框架，提出了一种结合切空间及特征空间校准的增量流形学习正则优化算法，得到适应不同环境的增量学习优化模型，提高学习效率。.项目在研期间负责人以第一作者身份撰写的论文发表于CCF推荐的A类顶级期刊TKDE，人工智能领域国际顶级学术期刊TCYB(SCI TOP期刊，CCF B，中科院一区，JCR一区，最新影响因子为11.079)，CCF C类著名国际学术会议PRICAI，SCI检索的国际学术期刊TST，CCF推荐的A类中文最顶级期刊《计算机研究与发展》等发表论文6篇，指导学术型研究生两名，毕业一名，项目组申请并授权专利一项，项目研究成果将为流式大数据的增量学习提供有效的理论支撑和良好的技术借鉴，实现高维数据流的高效学习，具有重要的理论意义和应用价值。