基于互联网的汉语-缅语双语平行语料抽取方法及语料库构建

For "the Belt and Road", we need to know well every relative country's culture and language in order to pave the way, among them, the research on Myanmar (Burmese) language machine understanding, information retrieval and machine translation of Chinese-Myanmar language, will promote the great significance for national strategy of "the Belt and Road". However, due to the Myanmar language belongs to the low-resources language, Chinese-Myanmar parallel corpus is very scarce, and there is a big difference of lexical and syntax between Chinese and Myanmar language. Combining with the characteristics of Chinese and Myanmar language, on this topic, we will research on the integration of topic context features and construction of word similarity characteristics for Chinese-Myanmar language by using bilingual LDA model, solving the problem on similarity calculation of bilingual comparable documents between the Chinese-Myanmar language. The research will base on bilingual lexicon, bilingual entity, and the method of parallel sub-sentential fragments on the bilingual comparable corpora of Chinese-Myanmar language. For the scarce problem of bilingual sentence alignment corpora for Chinese-Myanmar language, the research based on the method of bilingual sentence semantic representation of convolutional neural networks, realize the pivot as English language bilingual sentence alignment corpora for Chinese-Myanmar. Based on the above method to obtain the bilingual corpus comparable document, bilingual lexicon, bilingual entity, bilingual clause aligned fragments and bilingual sentence alignment corpora of Chinese-Myanmar Language automatically and the construction of parallel corpus for Chinese-Myanmar language. As the results of project research, there will be bilingual cross-language information retrieval and statistical machine translation of Chinese-Myanmar language that will provide the basis of language support for the research work.

“一带一路”需要语言铺路，研究缅甸语言机器理解以及汉语—缅甸语跨语言检索、机器翻译，对推动国家“一带一路”战略具有重大意义。然而，由于缅甸语属于低资源语言，汉语—缅甸语双语平行语料非常稀缺，并且在词法及句法方面缅甸语与汉语存在较大差别。结合汉语、缅甸语语言特点，本课题研究融合主题词上下文特征及词语相似度特征构建汉语-缅语双语主题模型的方法，解决汉语-缅语双语可比文档相似度计算问题，研究基于汉-缅双语可比语料的双语词汇、双语实体、双语对齐子句片段抽取方法；针对汉-缅双语句子对齐语料稀缺的问题，研究基于卷积神经元网络的双语句子语义表征方法，实现以英语为枢轴语言的汉-缅双语句子对齐语料获取。基于以上方法自动获取汉-缅双语可比文档语料、双语词汇、双语实体、双语子句对齐片段及双语句子对齐语料，构建汉-缅双语平行语料库。研究成果将为开展汉-缅双语跨语言信息检索、统计机器翻译等研究工作提供基础语料支撑。

缅甸语是典型的低资源语言，缅甸语分词、词性标记、句法分析及汉-缅双语平行语料构建是开展汉-缅机器翻译研究基础。项目围绕缅甸语词法、句法分析以及汉-缅双语平行语料库构建等内容开展研究，取得了以下成果：.（1）针对与缅甸语分词、词性标记方法，提出了基于神经联合模型的缅甸语音节分词、词语分词以及词性标记的方法；针对缅甸语依存句法分析面临语料稀缺问题，提出了基于迁移学习的缅甸语依存句法分析方法。针对汉缅双语词典构建问题，利用可比文档中上下文的信息，提出了基于半监督的汉缅双语词典构建方法和融合主题及上下文特征的汉缅双语词汇抽取方法，构建了汉-缅双语词典；针对汉缅双语文档、句对抽取任务，根据汉缅双语网站中可比文档的特点，提出了融合主题模型及双语词向量的汉缅双语可比文档获取方法；针对缅甸语字符组合特征导致缅甸语文字识别不准确的问题，提出了基于知识蒸馏的缅甸语OCR方法，解决了基于文本-图片跨模态的双语文本数据获取。针对双语平行句对抽取任务，提出了基于CNN-CorrNet网络的汉缅平行句对抽取方法，基于枢轴语言的汉语-缅语双语平行句对获取方法以及结构特征一致性约束的双语平行句对抽取方法；针对汉-缅双语平行语料稀缺导致机器翻译性能较差的问题，提出了基于多语言联合训练的汉英缅神经机器翻译方法。（2）课题研究成果发表相关论文11篇，其中SCI收录2篇，EI收录2篇，北大中文核心期刊7篇，受理中国发明专利9项，其中，授权发明专利2项，登记软件著作权5项，培养硕士研究生12名。（3）在语料库构建方面，构建了分词及词性标记语料库40万词，缅甸语OCR识别语料600万条，汉-缅双语实体对5万词条，汉-缅双语句对200万条、双语词典15万条、汉-缅双语可比文档40万篇。（4）研发了缅甸语分词及词性标记系统，缅甸语OCR系统，汉-缅机器翻译系统，研发的系统已在网信、军方、安全等领域运用，取得了较好的应用效果。