基于跨语言主题向量的双语篇章可比度量化研究

Quantifying comparability of bilingual documents has an important guidance value on both the translation equivalence mining and the cross-language information processing. Currently, the topic similarity calculation is the major measure for the comparability. However, several problems such as the low topic consistency and the poor domain adaptation occur due to the lack of internal structure relationship and the differences between document and domain knowledge. .In response to the problems mentioned above, the project will enhance the accuracy of quantifying the comparability degree of bilingual documents, which designs the cross-language topic representation model, by adopting the multi-level topic matching and the domain self-adaptation. The project will be carried out on following items: 1) To improve the generalization ability of the cross-language topic model, the study will merge bilingual word alignment and word representation into the cross-language topic model to build the cross-language topic representation model. 2) The multi-level topic matching model of bilingual documents will be proposed by researching the consistent topic representation document search method from source language into target language and adopting the semantic hierarchical clustering method on document fragments. 3) In order to enhance the domain self-adaptation for quantifying comparability of bilingual documents, the method of translation extraction and disambiguation will be studied under the guidance of topic representation, and the interactive promotion model of different granular translation resources will be researched. The research will provide a fundamental theory to build the cross-language topic representation model and a new idea to produce the large-scale, multi-level and multi-domain quantization of Chinese-English bilingual document comparability.

双语篇章可比度量化对互译等价对挖掘和跨语言信息处理有重要的指导价值。当前主题相似性计算是可比度量化的主要手段，然现有方法无法分析篇章内部的结构关系，且无法缓解领域知识的差异性，导致可比度量化面临多层次话题漂移和领域适应性偏差等问题。.针对上述问题，项目将进行跨语言主题向量模型设计，通过话题层次匹配和领域自适应，提高双语篇章可比度量化的精准度。重点开展：1）研究联合双语词对齐和词向量融入跨语言主题，构建跨语言主题向量模型，提升跨语言主题对开放域篇章的泛化能力；2）探讨源语言与目标语言主题向量一致文档搜索方法，对文档分割片段进行语义扩展聚类，实现双语篇章多层次话题匹配模式；3）探索主题向量指导互译词汇抽取和消歧方法，通过领域新知识发现和不同粒度互译资源交互学习，提高可比度量化的领域自适应能力。研究为跨语言主题向量构建提供理论基础，为大规模、多层次及多领域的汉英双语篇章可比度量化提供新思路。

双语篇章可比语料的构建降低了双语平行语料获取的困难程度，被不断应用于统计机器翻译、跨语言信息检索、跨语言情感分析等领域。可比度量化对可比语料应用具有重要的指导价值，现有主题相似性计算是可比度量化的主要手段，然现有方法无法分析篇章内部的结构关系，且无法缓解领域知识的差异性，导致可比度量化面临多层次话题漂移和领域适应性偏差等问题。针对上述问题，项目将进行跨语言主题向量模型设计，通过话题层次匹配和领域自适应，提高双语篇章可比度量化的精准度。重点开展：1）将双语词向量融入跨语言主题模型，构建跨语言主题向量模型，提高跨语言主题向量表征篇章的泛化能力；2）将互联网搜索的不同语言主题向量一致的文档，分割片段进行语义扩展和层次聚类，提高双语篇章多层次话题匹配的一致性；3）通过主题向量指导互译词汇抽取，扩充领域双语知识，构建不同粒度资源交互式学习模型，提升可比度量化的领域适应性。项目通过以上问题的研究，突破了基于跨语言主题向量的双语篇章可比度量化关键技术，达到以下目标：提升泛化性：将双语词向量融入跨语言主题，构建跨语言主题向量模型，提高模型的泛化能力，进行20万汉英双语篇章构建和可比度量化；优化一致性：利用双语篇章多层次话题分割、扩展和聚类，提高可比度量化精度，实现双语篇章的可比度高于0.8，其中，95%双语篇章的可比度高于0.9；增强适应性：利用主题向量指导领域新知识发现和多粒度交互作用，实现篇章可比度量化对不同领域的自适应性，涉及科技、经济、文化、法律、金融等10个领域。.本项目的研究，建立了双语篇章可比度量化试验系统，提出了一种大规模、多层次及多领域的汉英双语篇章可比度量化方法。项目研究过程中，取得了一些知识产权，主要包括发表学术论文4篇、申请专利3项、登记软件著作权3项。