基于门控卷积神经网络和长期记忆建模的复杂文本分类模型研究

Natural language understanding (NLU) is the core technology in the field of artificial intelligence, and it will act as a dominant method for human-computer interaction. Therefore, the research of NLU has great significance in theory and application. Many applications of NLU rely on text classification as it is a fundamental task to understanding topics of a message. Deep neural networks work in an end-to-end manner and require less human supervision, which make it possible for them become a general-purpose an powerful solution in deep semantic understanding and inference. However, the existing deep learning solutions are ineffective in terms of architecture design and collaboration mechanism. As a result, they can not effectively capture the hierarchical semantic representation and supporting evidences for the inference. This proposal studies the relationship between design of gates and gradient vanishing, the depth of convolutional nets, and multi-prototype word semantics; the relationship between the number of recurrent hops and the effectiveness of the long-term memory modeling and inference. In details: adapting a gate mechanism based convolutional net to transform the input text into the feature vector, which can avoid the problem of gradient vanishing in the convention neural net; putting forward in the depth of the net, which makes it possible to modeling the long-term memory and providing the mapping between the feature obtained and the memory component; obtaining the integrated feature by inserting, deleting, or updating memory slots in the memory component. Theoretical breakthroughs are expected to be achieved in deep semantics and inference. And the research results can be applied to various NLP tasks such as complex text classification, machine translation, and Q/A.

自然语言理解是人工智能领域里最核心的技术，即将成为新的和谐人机交互方式，具有重大的研究意义和应用价值。而分类技术对于判断用户意图至关重要。神经网络模型以其端到端数据驱动的特点规避人为特征工程，在构建通用语义理解和逻辑推理能力等方面具有巨大的应用潜力。但是现有的深度学习框架在实际操作中不能有效组织各个神经网络模块的协作，直接影响到提取关键特征的性能，继而无法进行准确推理并理解长依赖文本。本项目针对门控函数对梯度回流、卷积神经网络级数及多层次语义特征提取的关系机理开展研究，并基于这种多义语义对长依赖学习及逻辑推理机制开展研究：优化门控卷积层的设计，有效加深模型级数，使其能够更多地挖掘文本中的多层自然含义；使用记忆和注意力机制对输入文本的中间语义表示进行转换，从而准确获取长距离依赖特征，并应用于复杂文本处理中的主题、情感分类任务，智能对话系统中的意图理解、逻辑推理任务。

复杂文本分类技术旨在解决多标签分类、多伦对话系统中的用户意图识别、半监督文本分类等任务。本项目通过设计一种新颖的门控注意力机制，提升深度神经网络提取类别特征、发现上下文长距离依赖的能力；探索了提出方法在对话策略学习中的应用；提出基于生成对抗学习的无监督文本分类方法、文本与图像融合技术。提出方法有助于构建智能文本检索、人机对话系统、手语翻译与生成系统。本项目在国内外重要刊物上发表了 15 篇学术论文，申请知识产权6项，培养硕/博士生13人、青年科技工作者3人。