基于噪声分组和对抗训练的语音增强方法研究

The widely use of intelligent speech technology in mobile devices has brought new challenges for speech enhancement. The development of deep learning technology has provided new ideas for solving such problems. Against the main factors limiting the practical application of the speech enhancement methods based on deep learning, this project will carry out the research work in the following three areas: first, in order to improve the speech enhancement performance, based on the correlation between speech and noise signals in the time domain and frequency domain, a model will be established by using deep convolutional neural network to estimate and represent the speech components in the noisy speech, and the network structure optimized for the speech enhancement task will be constructed; secondly, for the problem of insufficient generalization ability of the model, according to the key factors influencing the generalization ability of the model, a speech enhancement model will be established based on adversarial training to minimize the impact of environmental factors on speech representation; finally, for the problem of model complexity caused by the diversity of the noise characteristics in the training set of a single model, a noise grouping method for noise type will be proposed and incorporated in the training process of the adversarial model, and a speech enhancement system based on noise grouping will be established. By improving the speech enhancement performance, improving the generalization ability and reducing the complexity, the research of this project will promote the application of speech enhancement method based on deep learning in real-time speech processing system.

智能语音技术在移动设备上的广泛应用给语音增强带来了新的挑战，深度学习技术的发展为解决该类问题提供了新的思路。针对制约基于深度学习的语音增强方法的实际应用的主要因素，本项目从三个方面展开研究：首先，为了提高语音增强性能，基于语音和噪声信号在时域和频域的相关性，采用深度卷积神经网络建立模型来估计和表示含噪语音中的语音成分，构建针对语音增强任务优化的网络结构；其次，针对模型泛化能力不足的问题，依据影响模型泛化能力的关键因素，建立基于对抗训练的语音增强模型来最小化环境因素对语音表示的影响；最后，针对单一模型下由训练集噪声特性的多样性带来的模型复杂度问题，提出针对噪声类型的噪声分组方法，并将其融入到对抗模型的训练过程中，建立基于噪声分组的语音增强系统。通过提升语音增强性能、提高模型泛化能力并降低模型复杂度，本项目的研究将促进基于深度学习的语音增强方法在实时语音处理系统中的应用。

基于深度学习的语音增强方法相比传统统计学方法显著提高了低信噪比和非平稳噪声下的语音增强性能；但是其在网络结构设计、泛化能力和执行效率上的不足制约了该类方法在实际语音处理系统中的应用。本项目围绕上述问题展开研究，研究了基于深度卷积神经网络的含噪语音特征表示，深入挖掘含噪语音的时频特征，结合不同类型网络的特点，提出了多种针对语音增强优化的网络结构，提高了深度神经网络的语音增强性能；研究了影响语音增强模型泛化能力的关键因素，提出了基于生成对抗网络的训练集噪声类型扩充方法，提高了语音增强模型的泛化能力；借鉴传统语音增强方法中的时频特征计算方法，从网络输入形式的选择、网络结构的设计研究了降低语音增强模型复杂度的方法，提高了语音增强模型的训练效率和语音增强的执行效率。通过本项目的研究，得到了具有语音增强性能更好、泛化能力更强和执行效率更高的深度神经网络语音增强方法，本项目的研究成果和研究经验可以促进深度学习语音增强方法在实时语音处理系统中的应用。