语音评价中的韵律建模和评价方法研究

Despite of the great achievement in speech assessment in recent years, the evaluation objects are still limited to the pronunciation quality, vocabulary, fluency of speech, etc., and the assessment of speech prosody remains inadequate and incomplete due to the lack of effective prosody model and prosody assessment methods. However, one's language ability cannot be fully reflected without the prosody manipulation skill. As a matter of fact, prosody is more important in language communication. Based on a large-scale corpus, this project will investigate the prosody modeling and evaluation methods from aspects of auditory perception and its role in language communication. The major work of this project are exploring the stable and non-stable textual and acoustical features of Mandarin prosody; building a prosody-chunk based prosody phrase prediction model; constructing a hierarchical Mandarin stress model based stress detection and evaluation model and discovering the intonation pattern then proposing a similarity based intonation assessment method. With these achievements, this project will further propose an overall assessment method by integrating the prosodic phrase, stress and intonation models. This research could acquire a better understanding of prosody generation in human speech, which is of great scientific importance to the speech understanding and cognition, and also provide a theoretical and practical basis to the development of Computer-assisted language learning system, spoken language assessment, Text-to-Speech system etc.

由于目前语音评价中缺乏有效的韵律模型以及韵律评价方法，现有语音评价多局限于发音质量、词汇以及流利度等方面的评估，对语音中的韵律评估还不够完善，无法真实反映学习者的语言能力。针对该问题，本项目将从听觉感知角度入手，建立适用于语音评价的大规模韵律标注语料库，结合韵律在语言交流中的重要作用，深入探索语音评价中的韵律建模和评价方法。重点从文本特征和声学特征层面探讨韵律的稳定与非稳定性特征；建立基于韵律组块的韵律短语预测模型；构建基于层级重音模型的重音检测和评价方法；挖掘语调稳定模式并提出基于相似性的语调评价方法。在以上研究基础上提出融合韵律短语、重音和语调的非线性韵律整体评价算法。本项目将有助于加深人们对韵律加工过程的理解，对促进言语理解和认知的研究具有重要科学意义，也能为计算机辅助语言学习系统、口语评测、语音合成等研究提供必要的理论和实践基础。

本项目将从听觉感知角度入手，建立包含陈述、疑问、感叹等不同韵律表现力的大规模韵律标注语料库，深入研究了语音评价中的韵律建模和评价方法。重点在大语料库的基础上，从文本特征和声学特征层面研究了韵律的声学特征和表现；并建立融合句法结构的韵律短语预测模型，以及基于增强字向量的韵律短语预测模型；对汉语连续语流中的时长约束关系进行了分析建模，并采用极限学习机的方法提高了汉语时长预测模型的准确率；构建了基于层级重音模型的重音检测、生成和评价方法；挖掘陈述、疑问、感叹等不同语调模式的特征，在语气识别任务以及语气生成任务上进行了验证。本项目的研究成果将有助于加深人们对韵律加工过程的理解，对促进言语理解和认知的研究具有重要科学意义，也能为计算机辅助语言学习系统、口语评测、语音合成等研究提供必要的理论和实践基础。