异质噪声场景语音识别中的结构化深度学习研究

Noise robust speech recognition is one of the unresolved key technologies in the field of speech recognition. Focusing on the real "heterogeneous" noise data under the real scenario, the innovative research is conducted in this project. By focusing on solving the "mismatch between training and testing" problem in the "heterogeneous" data environment, this project will start from the noise modeling and acoustic modeling. The structured deep learning based methods are both developed on these two aspects, and the mechanisms of the human ear’s hearing and human’s environmental perception are also combined. The new theories and methods are explored to better utilize the “heterogeneous” noise data for robust speech recognition, compared to the previous methods, and this is a completely new research strategy. This research mainly includes these aspects shown as below: (1) Structured deep learning based noise modeling under the heterogeneous noise data, to explore the discrimination and relevance among the noise by the study of the noise itself. Mainly including the noise representation, classification and parameter estimation. (2) Structured deep learning based acoustic modeling under the heterogeneous data scenario, to deal with the “training and testing mismatch” problem in the heterogeneous data environment through the structured deep learning. Mainly including the structured deep modeling based speech denoising and robust feature extraction, structured environment-aware acoustic modeling and learning algorithm, noise adaption and joint optimization combining with a prediction-feedback mechanism. It is hoped that recognition accuracy and robustness of the system can be greatly improved through this study. This study has important theory significance and practical value.

复杂噪声场景下的鲁棒语音识别是语音识别领域尚未解决的关键技术之一。本课题着眼于真实噪声数据“异质性”所引起的“训练与测试失配”现象，从噪声模型的建模和声学模型的建模入手，均采用结构化的深度学习方法，并结合人类感知过程和人耳听觉机理，合理利用异质环境噪声数据，探索抗噪语音识别建模新方法和新理论，相比前人方法，这是一条新的研究思路。主要研究内容包括：（1）异质数据下噪声模型的结构化深度学习，通过对环境和噪声本身的研究来探寻噪声之间的区分性和相关性，包括对噪声的表达，分类及参数估计。（2）异质数据下声学模型的结构化深度学习，通过结构化的深度模型来应对异质数据下面临的“训练与测试失配”问题：包括结构化的语音谱除噪和抗噪特征表示，结构化的环境感知声学建模与学习算法，噪声自适应及结合预测反馈机制的联合优化方法。希望通过本研究可以较大地提升系统识别精度和鲁棒性。该研究具有重大理论意义和实际应用价值。

复杂噪声场景下的鲁棒语音识别是语音识别领域尚未解决的关键技术之一。本课题着眼于真实噪声数据“异质性”所引起的“训练与测试失配”现象，从噪声模型的建模和声学模型的建模入手，均采用结构化的深度学习方法，并结合人类感知过程和人耳听觉机理，合理利用异质环境噪声数据，探索抗噪语音识别建模新方法和新理论，相比前人方法，这是一条新的研究思路。本课题主要研究内容包括：（1）异质数据下噪声模型的结构化深度学习，通过对环境和噪声本身的研究来探寻噪声之间的区分性和相关性，包括对噪声的表达，分类及参数估计。（2）异质数据下声学模型的结构化深度学习，通过结构化的深度模型来应对异质数据下面临的“训练与测试失配”问题：包括结构化的语音谱除噪和抗噪特征表示，结构化的环境感知声学建模与学习算法，噪声自适应及结合预测反馈机制的联合优化方法。. 在项目执行过程中，我们提出了若干种结构化的创新方法，均有效地改善了语音识别系统在噪声场景下的识别性能。具体包括如下创新方法：1）极深卷积神经网络及自适应方法；2）基于神经网络建模的环境因子分析与表示；3）基于多因子环境感知的抗噪鲁棒语音识别；4）基于未来因子的语言模型建模与预测能力提升；5）复杂异质数据下的鲁棒端点检测算法；6）基于排列不变性训练的多人说话混叠语音分离与识别；7）基于深度生成对抗网络的数据扩充和抗噪建模；8）基于端到端模型的多人说话混叠语音分离与识别。利用以上这些方法，在抗噪语音识别基准测试集合Aurora4上，我们取得了目前报道的最优性能。基于本课题研究，发表了一系列高水平文章，相关算法也在真实系统中应用上线。. 本项目的研究思路和研究成果，对指导深度学习更好地建模，有强有力的指导和借鉴意义。相关方法和思路可以扩展到智能语音的其他相关任务中去，研究具有重大理论意义和实际应用价值。