大数据环境下的空间聚类方法研究

Spatial clustering is an important part of spatial data mining. The goal of spatial clustering is to find hidden pattern or similar regions from spatial databases. With the high speed development of information technique for spatial data, big data will undoubtedly bring many new challenges to the spatial clustering research. The proposal aims to deal with spatial clustering on big data, and details are listed as follows: (1)Based on the parallel computation framework, MapReduce, we firstly try to transfer existing spatial clustering methods to the distributed computing platform, and check whether they would work. This transformation is to satisfy the requirement of the scientific computing on big data. And, if this works, the output could be used to support the spatial and temporal-spatial clustering research on a data set with bigger size and higher dimension. (2)We firstly introduce LDA model to propose an on-line spatial clustering method. The method could satisfy the performance requirement from big data processing, and it also avoid some drawbacks caused by spatial data, like high dimension and lots of noise. This work will definitely support the research on temporal-spatial mining and trend prediction. With big data, good data mining techniques could help researchers explore temporal-spatial patterns easily and freely, and help them understand the complicated change of Earth more and deeper. In general, the research in this proposal could help related researchers find more latent association and change rules from the attribute set. And this is very novel and important in both theoretical research and real-world application.

空间聚类是空间数据挖掘的一个重要分支，目的是从空间数据库中发现隐藏的模式或识别出相似的地域。随着空间信息处理技术的高速发展，空前规模的大数据势必给聚类研究带来新的挑战。本项目的研究面向大数据环境下的空间聚类方法展开，内容包括：(1)利用并行计算框架MapReduce，首次将现有聚类算法移植到分布式计算平台,以满足空间大数据的科学计算需求，也为更大规模、更高维度的空间数据及时空数据聚类研究提供了基础；(2)首次利用LDA模型设计了在线的空间聚类算法，不仅满足了大数据的并行计算需求，还解决了维度高、噪声多等问题，为时空信息挖掘及趋势预测等提供了研究基础。在大数据环境下，先进的数据挖掘技术可以让空间领域的科研工作者更轻松地探索数据的时空模式，不断深化对地球系统复杂演变过程的理解。而本项目的研究内容对于更全面地发现空间数据属性间存在的潜在联系和变化规律，具有重要的理论意义和应用前景。

本项目的工作是面向大数据环境下的空间聚类技术研究，研究内容包括：为探讨传统聚类方法如何被扩展、移植到并行计算框架中，基于分布式计算框架Hadoop搭建了空间大数据的存储及聚类分析平台；深入研究了传统聚类算法、特别是K-means算法基于分布式计算框架MapReduce的实际应用情况；并针对多核处理器、耦合CPU-GPU结构下的大数据挖掘进行了详细分析；设计并实现了基于LDA主题模型的空间聚类算法，分析该算法在处理空间大数据时的优劣表现，进而基于深度学习及迁移学习方法，提出一种全新的基于Siamese网络的空间图像聚类算法，以解决训练数据匮乏、需人工提取特征等问题带来的挑战，以期为空间大数据的挖掘分析提供一种新的思路和技术。