基于深度神经网络的说话人分离和识别算法的研究

Recently, speaker recognition technology has become steadily more present in everyday life, as seen by the proliferation of applications including bank and mobile personal agents, etc. Performance of these systems, however, degrades significantly in the presence of background noise and other speakers. Compensation for noise typically involves preprocessing the acoustic signal to separate the speech signal of a target speaker from these mixtures, and then feeding this processed input into the recognizer. This project seeks to make Deep Neural Network (DNN) more resistant to noise by integrating speech separation and speaker recognition. The first research area seeks to estimate the space features of the speaker, and explore speaker unsupervised separation and supervised separation algorithms with the space features and the acoustic features on the base of DNN. The second area seeks to select the data for training the DNN to extract speaker’s deep features. The final research area examines ways for speaker recognition based on DNN. With the focus of integrating speech separation and speaker recognition, the project will train the separation and recognition systems in an integrated manner so that the speech feature of the recognized speaker can inform the separation. This has great theoretical significance and practical value for speaker recognition.

近年来，说话人识别技术在人们日常生活中的应用日趋广泛，在银行、移动业务处理等领域具有广泛的应用前景。但在噪声干扰环境中，由于噪声和其他说话人声音的干扰，说话人识别系统的性能会显著下降。为提高说话人识别系统的性能，须对说话人的声音信号做预处理，如将声音信号先分离出来，然后再识别。本项目主要研究基于深度神经网络(Deep Neural Network，DNN)的说话人分离和识别的算法。研究基于DNN的说话人空间特征的估计算法，利用说话人的空间特征和声纹特征，研究说话人无监督和有监督的分离方法；研究DNN特征提取器训练数据的构建方法以及说话人深度特征的提取算法；研究DNN说话人判决模型的设计方法。将说话人的分离和识别集成一整体，利用声纹特征提高分离系统的性能，联合优化整个系统，为说话人识别技术的实用化提供具有理论意义和实际价值的研究成果。

本项目针对复杂声场环境下，说话人有监督和无监督分离和提取、声纹深度特征的提取和融合、说话人模型构建等挑战性问题进行了深入的研究。主要内容包括：1) 提出了深度最近邻聚类算法实现语音混合信号的盲分离，并且设计了一种新的深度聚类自动编码器网络结构来实现说话人的无监督分离；针对共信道说话人识别，提出了一种基于时域卷积网络的目标说话人提取算法。2) 利用解析稀疏模型，提出了说话人的长时声学特征选取的方法；提出了基于独立向量分析和并行卷积神经网络的语音特征融合算法；基于门控并行卷积神经网络，提出了多尺度频域特征的提取和融合方法。3) 提出了基于语音信号稀疏表示的说话人的模型构建方案，并采用深度神经网络模型构建说话人识别分类器。提出了基于谱图增强和对抗学习的鲁棒说话人识别算法。课题组已发表学术论文9篇，其中SCI期刊论文5篇，EI论文1篇，申请中国发明专利2项。