基于深度学习和主题模型的文本特征提取方法研究

Deep learning has been successfully applied to a variety of pattern classification problems, such as: speech recognition, image processing. However, there are few reports concern about text feature extraction. On the other hand, topic model has been widely used in text feature extraction. But most of these models did not consider contextual information and space structure of the text information. Therefore, the study of our project is as follows: 1. Construct deep learning model for text feature extraction and propose a new deep learning sparse network topology structure and learning algorithms for massive text features extraction; 2. Merge context and semantic relationships together into the topic model, construct semantic text feature extraction method based on topic model; 3. Construct deep learning feature extraction model on topic space, study the fast and effective integration strategy with the linear transformation from vector space model to topic space and deep learning non-linear transformation feature extraction procedure; 4. Construct feature extraction evaluation algorithm, make comparisons among the new proposed algorithms and classical algorithms and find out the scope for each algorithms and give out the recommendations for their further usage. 5. Construct text feature extraction platform based on the topic model and deep learning. The implementation of this project will promote the rapid development and application of the deep learning and the topic model, the study will provide new and efficient algorithms and models for text feature extraction.

近年来，深度学习已成功应用于多种模式分类问题并在语音识别，图像处理等方面取得了深远的影响。但在文本特征提取领域则罕见报道。另一方面，主题模型在文本特征提取方面取得广泛的应用。然而，却没有将文本的上下文信息和空间结构信息融合进来。因此，本项目研究内容如下：1.构建深度学习的文本特征提取模型，研究新的适用于海量文本特征提取的深度学习稀疏网络拓扑结构和学习算法;2.将上下文环境以及语义关系于主题模型相融合，构建语义主题模型的文本特征提取方法;3.构建主题空间深度学习特征提取模型，研究由向量空间模型到主题空间的线性变换与深度学习非线性特征提取流程的快速有效的融合; 4.构建特征提取评估算法，将新提出算法与经典算法进行比较分析，给出各种算法的适用范围和使用建议; 5.基于主题模型和深度学习的文本特征提取平台搭建。本项目的实施将推动深度学习和主题模型的快速发展和应用，为文本特征提取研究提供新的高效算法

本项目研究了深度学习在文本挖掘中的应用，重点研究了使用DBN深度学习方法实现对高维稀疏文本向量空间模型的降维，然后利用KNN或SVM分类算法对降维数据进行分类。提出一种解决大规模旅行商问题（TSP）的层次求解方法，把原大规模TSP问题转成化广义旅行商问题（GTSP），把求解大规模TSP问题转化为求解GTSP问题和几个小规模TSP问题。提出了一种基于多重离心机模型的特征选择方法，完成了基于多重离心机模型的特征选择方法研究。在原有的SAE算法中引用了一种新的激活函数ReLU，来代替传统的Sigmoid激活函数和Tanh激活函数，提出了基于ReLU激活函数的SAE算法的文本分类研究。针对深度自编码神经网络学习算法在形成特征表达的过程中会调节优化大量参数，从而使网络训练时间加长、BP算法陷入局部最小等问题，将逐层贪婪初始化思想（Layer-wise）引入到稀疏滤波的学习算法中，提出了基于深度稀疏滤波的文本分类算法。