多源数据挖掘的关键技术研究

Big data has been widely applied in many real-world domains, and achieved howling successes during the past years. Most of investigations about massive data have been done more on storage, retrieval and management, while less on analysis and mining, which need more efforts to be put. This project will investigate the key technologies of multi-source data mining. Specifically, we will take multi-source data as our studying objects, and investigate the topic of new mechanisms of multi-source data mining according to the characteristics and applications of multi-source data. The emphasis of this project will be placed on proposing new theories and methods of outlier detection, dimension reduction, formal representation and information extraction for multi-source data, by using the techniques of statistical learning, such as k-nearest neighbors, local weights, canonical correlation analysis, low-rank factorization and regularization factors. The purpose is to illustrate the impacts of data quality on multi-source data mining, find the related facts to construct models, discuss the advantages of statistical learning on model construction, explore the mechanisms and key techniques of multi-source data mining, put forward efficient and effect mining models and methods, so that a whole theoretical framework of multi-source data mining is formed, after an application has been presented. The study of this project will improve the mathematical theories of data quality, mining techniques and even the applications of multi-source data. Furthermore, it can also provide some useful suggestions, beneficial explorations, practical and feasible solutions for the real applications of massive data.

大规模数据在现实许多领域中已得到广泛应用，并取得了令人瞩目的成就。现有大规模数据的研究工作主要体现在数据存储、检索和管理方面，而分析和挖掘等相关技术尚未成熟，有待深入研究。本项目以多源数据为研究对象，从多源数据结构及特点出发，围绕多源数据挖掘新机制这个主题展开研究，重点建立多源数据的异常检测、维数约简、形式化表示和信息获取等新理论和新方法，通过采用近邻、局部权重、典型相关分析、低秩分解和正则化等统计学习技术，阐明统计学习对模型构建的优势，探讨数据质量对多源数据挖掘的影响，明确影响模型构建的相关因素，揭示多源数据挖掘的理论及关键技术特点，建立高效率和高性能的多源数据挖掘模型，给出多源数据的应用案例，形成多源数据挖掘的完整理论体系。本项目的研究对于提高多源数据质量、完善挖掘技术的理论研究和实际应用具有重要意义，可为大数据的真实落地提供有益的探索和切实可行的解决方案。

多源数据是大规模数据的表现方式之一。数据质量的提升对于大规模数据的挖掘起着关键作用，也是大数据分析的热点研究方向。本项目以多源数据为研究对象，从多源数据结构及特点出发，围绕多源数据的质量提高这个主题展开研究，重点建立多源数据的异常检测和维数约简等新理论和新方法，通过采用近邻、矩阵分解和稀疏学习等统计学习技术，阐明统计学习对模型构建的优势，探讨数据质量对多源数据挖掘的影响，建立高效率和高性能的异常检测模型。项目具体研究内容包括：针对数据规模大的问题，采用矩阵分解、谱聚类和随机游走等技术，提出哈希学习方法，提升数据检索效率；针对数据质量提高问题，通过局部投影策略、稀疏学习和随机森林技术，结合信息熵和领域稀疏度概念，提出了适用于大规模数据的异常检测方法；根据多源数据中数据相关的特点，利用多变量分析技术，度量多源数据的相关性，配合正则化因子，实现维数约简目的，最终构建可解释性、高性能的多源数据分类学习模型。