非均衡概念漂移网络舆情大数据流挖掘模型、算法与评价机制研究

Mining internet public opinion becomes a new research issue in the area of data mining. The previous study is in context of the static and balanced big data streams distribution, consequently the traditional theories and methods have troubled by previously unknown difficulties and challenges when faced with the imbalanced, dynamic and abrupt public opinion big data streams. Because this project is to generalize the conventional public opinion mining technology from the ideal state to the reality state, it will inevitably be extended into the studies line of imbalanced and big data with concept drift. In the scenario of the imbalanced data streams of internet public opinion, we intend to research systematically the technologies of public opinion mining be accompanied by concept drift with main contributions as follows: 1) To innovate the ensemble classifiers model for the imbalanced internet public opinion big streams with concept drift;2) To construct the ensemble method for the multiple base classifiers with time stamp; 3)To propose resampling methods for the base classifiers in the ensemble classifiers model；4)To put forward the transformation model from the imbalanced concept drift big data flow to the appropriate scale; 5)To reveal the coupling mechanism for the concept drift and dynamic theme evolution; 6)To propose evaluation mechanism and validation evaluation methods for the ensemble classifiers model for the imbalanced big data streams with concept drift. The significance of this project can be summarized as follows: The project aims to explore new ways for solving knowledge representation and system construction for processing dynamic and big data streams, with the main thread of "Classifier ensemble (model) - Large data stream reduction and sample (instance) - Dynamic thematic evolution (semantic) - Multi-measure measurement (evaluation)",and reveals the mechanism and laws in mining imbalanced data streams with concept drift, such that is will further enrich the technologies, methods and applications for mining imbalanced big data sets and data streams with concept drift. Furthermore, the study has great theoretical significance and far-reaching practical value for mining imbalanced big data streams with concept drift in the real situation.

网络舆情挖掘日渐成为研究热点。但先前研究基于静态、均衡分布、规模适量情境，面对非均衡分布、突变、主题演化的大数据流，传统理论和方法遇到了一定困难和挑战。本研究旨在将传统挖掘技术从理想态向现实态的泛化更新，故必导致向非均衡概念漂移大数据流路线的拓展延伸。研究主要问题有：1）构建非均衡概念漂移大数据流集成分类器挖掘模型；2）研究基于时间戳的基分类器动态权重集成方法；3）构建非均衡概念漂移大数据流向适度规模数据流的平滑转化算法；4）构建该模型内涵的采样方法；5）研究概念漂移与动态主题演化时序耦合规律；6） 构建挖掘评价机制及验证方法。本项目旨在探索非均衡概念漂移情境下“分类器集成（模型）—大数据流精简及采样（样本）—动态主题演化（语义）—多测度衡量（评价）”具有系统逻辑的知识表征与系统建模新路径，揭示非平衡概念漂移大数据流与动态主题演化的融合挖掘机理，对舆情数据流挖掘具有一定理论意义与实践价值。

本研究突破国内外非均衡网络舆情数据流局限于学习方法和演化机制的传统路线，着眼于提升复杂情境下大数据流学习速度和提升分类精度，另辟蹊径，构建非均衡网络舆情数据流学习算法和分类模型。我们已按照研究计划要求完成研究计划要点，具体包括：（1）研究了非均衡概念漂移大数据流集成分类器挖掘模型；（2）研究基于时间戳的基分类器动态权重集成方法，并针对IoT 环境下社交网络模型作了研究；（3）构建异质信息网络（heterogeneous information networks）中非均衡概念漂移大数据流挖掘模型；（4）基于开放网络信息源的实体挖掘方法研究、岭回归在动态学习权重矩阵中的应用以及智能感知分析三个方面展开技术方法的研究，进一步挖掘动态主题演化时序耦合规律和评价模型。围绕本研究方向，已出版学术著作2部，发表学术论文26篇，其中，SCI/SCIE期刊论文8篇，EI期刊论文2篇，CSSCI及中文核心期刊论文11篇，圆满完成项目预期研究目标。