基于深度学习的句子相似度计算研究

The existing sentence modeling methods based on deep learning focus primarily on the construction and improvement of deep neural network, in which too much emphasis is on automatic learning of the feature representation and abstraction, whereas the full-fledged systematic expert knowledge is not considered at all. Our project will focus on the key techniques of deep learning based sentence similarity calculation and its application to machine translation the languages of Chinese and Japanese. Based on the study of the characteristics of deep neural network and human cognitive process, we aim to improve the performance of sentence similarity calculation from the following three perspectives: firstly , we propose a word concept generalization method based on the word semantic computing, in which, the generalization granularity and standards of the semantic unit and the designing of generalization algorithms are our focus. We intend to integrate the expert knowledge into our generalization rules to improve learning ability of sentence patterns and sentence embedding. Secondly, a new sentence embedding model based on deep learning is proposed, in which sentences will be treated as a hierarchy of “word-phrase-sentence” and the improved mechanisms of LSTM and Attention are used to build the deep learning network. Thirdly, a rational and efficient algorithm to compute the similarity between sentences will be designed, in which, the sentence similarity calculation will be considered simultaneously in the sentence modeling so that a global tuning of the parameters in both processes will be reached.

现有的基于深度学习的句子建模方法，主要侧重构建和改进深度网络框架，过于强调自动学习特征表达和抽象的过程，没有将体系完善的语言知识充分融合到学习框架中。本项目从汉语和日语两种语言入手，研究基于深度学习的句子相似度计算的关键技术并将其应用到机器翻译等领域。针对深度网络模型及人类对句子认知过程的特点，从三个方面提高句子相似度计算的性能：一是提出一种基于语义计算的单词概念泛化方法，研究语义单元的泛化粒度、泛化标准及泛化算法，将语言知识融合到泛化规则中，提高学习句子模式的能力和嵌入表示句子性能；二是提出一种基于深度学习的句子表示模型，将句子视为“词－短语－句子”的层级结构，综合考虑LSTM和Attention改进机制构建深度学习网络模型；三是设计一个合理、高效的算法来计算句子之间的相似性，主要考虑将相似度计算算法加入到句子建模过程中，达到句子建模过程和相似度计算过程的参数可全局调优的效果。

围绕句子相似度计算的若干关键问题展开了深入研究，主要研究内容包括词表示与词法分析、双语平行句对语言资源建设、短语识别及短语相似度计算、句法分析与句子相似度计算、句子相似度计算的应用研究等，取得研究结果如下：.（1）提出一种基于语义计算的单词概念泛化方法，研究语义单元的泛化粒度、泛化标准及泛化算法，将语言知识融合到泛化规则中，提高了嵌入表示句子性能；（2）提出一种基于深度学习的句子表示模型，将句子视为“词－短语－句子”的层级结构，设计实现了一种基于嵌入表示的相似度计算算法，显著提高了相似度计算的性能；(3)建立了基于词表示与词法分析、短语识别与泛化、句子表示与句法分析的句子相似度计算模型，并将其应用到机器翻译和机器阅读理解任务中，有效提高了机器翻译和机器阅读理解的性能。.研究结果表明，句子相似度计算方法对提高句子级的自然语言处理任务有显著的作用，另外，通过基于篇章的汉语自动分词研究，发现利用篇章的上下文信息可以消除部分句子级无法解决的语义歧义问题，基于篇章信息的自然语言处理研究有重要的意义。