基于语音增强的鲁棒性语音识别方法研究

Improving the noise robustness of the speech recognition system in the actual environment is one of the key difficulties and research hotspots of practical speech recognition. However, because the statistical characteristics of speech and noise signals are extremely complicated, traditional single-channel speech enhancement need to adds a lot of constraints for the convenience of theoretical derivation, so the enhancement effect is limited to a large extent, and it is difficult for the enhanced speech to directly improve the recognition performance. With the successful application of deep learning in the field of speech recognition and enhancement, it is expected to make up for the shortcomings of traditional noise robust methods..This project aims to make full use of traditional and deep learning-based single-channel speech enhancement methods in noise suppression and speech retained with their own advantages and disadvantages. For example, we can not only use the advantages of deep learning-based speech enhancement algorithms with large amount of training data to obtain the prior information, but the advantage of the traditional single-channel enhancement algorithm adaptively adjusting the estimated parameters for test data can use utilized. By combine the both advantages, we can establish a set of research schemes that use the combined signal preprocessing system as the front-end for speech recognition to make the speech recognition system obtain best recognition performance under adverse environments. At the same time, some of the research results of this project are also significance to fundamental issues in the field of signal processing such as speech enhancement.

提高语音识别系统在实际环境下的噪声鲁棒性是语音识别实用化的关键难点和研究热点之一。但由于语音和噪声信号的统计特性都极其复杂,传统单通道语音增强为了方便理论推导加了许多限定条件，因此很大程度上限制了增强效果，从而增强后的语音很难直接提升识别性能。随着深度学习在语音识别和增强领域的成功应用，有望弥补传统噪声鲁棒性方法的缺陷。.本项目旨在充分利用基于传统和深度学习的单通道语音增强方法在噪声抑制和语音保留具有各自的优点和缺点，比如利用基于深度学习语音增强算法可以使用大量数据获得先验信息的优势和利用传统单通道增强算法对于测试数据可以自适应调整估计参数的特点，建立起一套将两者结合的语音信号预处理系统作为语音识别前端的研究方案，以期语音识别系统能在实际噪声环境下获得最佳识别性能。同时，本项目部分研究成果对语音增强等信号处理领域的基础问题也具有重要意义。

语音识别是实现智能人机交互的关键技术之一，但在实际复杂场景下，由于各种干扰带来识别率的大幅降低，用户体验并不好。语音识别的噪声鲁棒性研究始于上世纪80年代，虽然近年来基于深度神经网络的语音增强方法已经展示出较好的性能，但在信噪比极低的情况下，这个问题始终没有得到很好的解决。本项目基于近年来深度神经网络在语音领域和视频领域的成功应用，从如下方面展开了基于深度学习的噪声鲁棒性方法研究。首先，根据传统语音增强算法和深度学习方法各自优缺点，建立起一套将两者结合的语音信号预处理系统作为语音识别前端的研究方案。其次，探索基于音视频的语音增强及分离系统，将视频信息引入分离系统中，有效缓解单模语音被噪声污染性能下降的问题，从而可以达到更好的识别效果，在科大讯飞内部真实测试集上表现出优异的性能。此外，我们还探讨了基于预训练的音视频对齐网络辅助多模分离网络，该技术可以有效利用海量真实数据。此项目的成果已经在科大讯飞公司车载和地铁项目中成功落地，并准备扩展到法院、智能机器人、录音笔和智能家居等项目中，有着非常广阔的应用前景。