基于深度学习的新型生成式摘要模型研究

The information on the Internet, already massive, is still growing at an exponential pace. To alleviate the information overload, there is a pressing need for the development of advanced summarization tools. The current state-of-the-art of machine summarization approaches is extraction-based, which selects salient sentences from the original text and concatenates them directly as a summary. Due to its nature, an extractive approach guarantees to produce a summary with good grammatical sentences. However, an extractive summary is far from expected. Coherence is often problematic especially when summary sentences are extracted from different sources. Also, non-essential or redundant information in selected sentences is common. Although additional compression may be admitted, it causes further readability issue..An ideal way to summarize is to create new sentences with good readability and coherence to reorganize the salient information in the original text. This process resembles human summarization strategies and is known to be challenging for machines. It requires sophisticated techniques for natural language understanding and generation, which themselves are growing fields. Previously, only a limited number of knowledge-driven approaches have been developed in specific domains and with hand-crafted templates/rules. In contrast to extraction-based approaches, research efforts along this line are referred to as abstractive or generation-based approaches with particular emphasis on generating new sentences. We believe that the challenge of generation-based summarization deserves renewed attention with remarkable success of neural network models in machine translation and machine conversation lately..The goal of this project is to develop data-driven neural network models for generation-based summarization by exploring their potentials for content representation, content focus identification and text generation. It relies on the design of effective network architectures and interaction mechanisms to “teach” machines to “learn”, to “summarize” and to “speak” the natural language from both semantic and structural views. As the first attempt to develop neural summarization machines, our research focuses on the following fundamental issues in the context of summarization: derive effective word/sentence/document latent semantic representation learning approaches according to the structure information; learn to discover the focused content in either supervised or unsupervised manner; and plan document-level and sentence-level content collaboratively in order to generate abstractive summaries..While this project aims to explore a new framework for summarization based on neural machines, it not only facilitates the advanced research in generation-based summarization but is also beneficial to the related research fields like question-answering and contributes to the development of the intelligent machines that are equipped with the effective language skills.

大数据时代信息严重超载，为迅速获取有效信息带来障碍，因此智能文本摘要的方法研究和系统开发具有广阔的应用前景。尽管自动文摘已有半个多世纪的研究历史，由于现有语句抽取式和模板填充式摘要方法的内在局限性，机器摘要和人工摘要之间的差距尚无法逾越，开发新型摘要模型乃当务之急。受人工智能、深度学习和神经网络模型的最新研究成果鼓舞，我们大胆提出全自动生成式摘要的目标。我们立足于语言文字的独特性，以摘要为目的，研究文本理解和文本生成中所遇到的相关困难和关键问题。我们致力于分析、设计和实现数据驱动的神经网络框架及模型，包括探讨文本理解和生成的层级建模和协同学习机制、焦点信息的关注机制、模型自我完善等机制，及其相应策略和算法，以应对文摘全自动生成所带来的挑战，并在此基础之上开发相应的系统和应用。该项目的研究内容具有重要的研究价值和深远的学术影响。针对某些关键技术的前期研究结果亦表明，该项目的实施切实可行。

大数据时代信息严重超载，为迅速获取有效信息带来障碍，因此智能文本摘要的方法研究和系统开发具有广阔的应用前景。该项目着眼于生成式自动文本摘要研究，预期研究目标为：抛弃人工构建的特定领域摘要模板，借鉴人工智能、深度学习和神经网络模型的最新研究成果，开发数据驱动的文本摘要生成方法和模型，推进实现为机器赋予智能语言（包括语言理解和语言生成）能力以及信息摘要能力的理想目标。研究工作进展顺利，目标顺利达成，并且在人工智能热潮的带动之下有效延伸，为更加前沿和更具应用价值的未来研究项目申请做好铺垫和基础准备工作。..　　该项目以深度学习算法和神经网络模型在自然语言理解、自然语言生成及其相关应用中的深度探索为目标。项目实施期间，我们分别基于不同神经网络模型的架构以及不同文本摘要任务的特性进行了如下研究工作，主要包括：利用卷积神经网络模型的特征抽取优势，创新地把分类任务和自动文摘任务想结合，针对不同主题类型的文档集获取不同特色的摘要；充分利用受到普遍关注的注意力机制，联合学习关键信息聚焦和句子重要性排序，开发基于询问的自动文摘模型；在编码器和解码器架构上开发递归神经网络模型，利用复制机制和改写机制设计了两个不同的解码器实现新闻标题生成任务；开发事实感知的神经网络模型，利用双关注机制解决自动生成文摘中“假造”事实的问题；融合外部预训练语言模型，增强神经摘要模型的语言表达能力，并且减轻内部解码器承担语言模型的负担；特别关注不同信息之间的关联，进一步利用图神经网络模型为结构化信息建模，实现基于文本层次化结构以及基于事件结构化信息的多文档信息融合式摘要生成模型，等等。..　　至今为止，该项目共培养博士毕业生四名，另有一名学生刚刚提交论文，在读博士生一名，博士后一名，助理研究员七名。各位学生和研究人员认真投入，积极合作，迅速掌握国内外学术动态和领先技术，并探索可能的创新之处，经过努力收获令人满意的可喜成果，达到预期之人才培养目的。项目取得的研究进展和科研成果亦十分令人满意。至今，项目组成员共发表八篇期刊论文和十七篇会议论文，其中，七篇论文发表于富有极高声誉的IEEE和ACM会刊，十四篇会议论文发表在自然语言处理、人工智能、信息检索的顶级会议（包括ACL、COLING，AAAI、IJCAI、ICLR、SIGIR、CIKM等）。.