基于分布式麦克风阵列的多说话人跟踪方法研究

Speaker tracking involves determining and following the positions of speakers using distributed microphone array under some acoustic environments. The demand for robust speaker tracking is sharply increasing in many application areas, such as multimedia conference system, human-machine interface, automatic camera steering for surveillance, robot navigation, smart home, military, etc. .This project mainly researches the multi-speaker tracking methods using distributed microphone arrays, which includes the following contents: (1) The decision criteria based on the Gerschgorin circle theorem is presented for determining the number of speakers; (2) Based on the characteristic of the speech signal, the improved high resolution spatial spectral estimation is used to localize the multiple speakers; Moreover, considering the estimation error of microphone calibration, a robust time delay estimator is developed, and then the total least squares technique and the maximum likelihood theory are used for speech source localization; (3) Considering the diversity of speaker motion models and characteristics of distributed arrays, the distributed nonlinear Kalman filtering and Bayes recursive filtering, and data fusion algorithms which were often used in the multi-sensor multi-target tracking, are exploited to solve the multi- speaker tracking problem; (4)The multi-speaker tracking system is characterized by modeling multi-speaker number, state and observation as random finite sets, and then the Bayesian set-valued estimation problem is solved by multi-Bernoulli probability hypothesis density filter, thus can jointly estimating the number of speakers and their states. (5) A real-time multi-speaker localization and tracking system based on distributed microphone arrays is constructed on the high speed DSP platform..The object of the project is to further improve the multi-speaker localization and tracking performance based on some novel methods as above mentioned, and to build a high performance real-time multi-speaker tracking system which can effectively track multiple speakers under adverse acoustic environments.

本项目研究基于分布式麦克风阵列的多说话人跟踪方法，它在多媒体通信、人机接口、机器人、军事等领域有广泛应用价值。主要研究内容包括：(1)应用矩阵理论中的盖氏(Gerschgorin)圆估计方法，构造语音信号源数的判别准则，进行说话人数目的有效估计；(2)根据语音信号特点，应用高分辨率空间谱估计理论，研究分布式多说话人声源定位方法；研究稳健的时延估计技术，并用总体最小二乘和最大似然理论，研究多说话人定位方法；(3)考虑到说话人运动模型，应用分布式多传感器多目标数据融合技术，采用分布式非线性卡尔曼滤波和贝叶斯递推滤波，通过Gossip算法，研究分布式多说话人跟踪方法；(4)基于随机有限集合理论，将说话人状态和说话人数目统一用随机有限集表示，应用贝叶斯集合滤波技术，结合分布式一致性融合策略，研究说话人数目变化情况下多说话人跟踪方法；(5)建立基于分布式麦克风阵列的多说话人定位与跟踪实时处理系统。

基于分布式麦克风阵列的多说话人跟踪技术是语音处理领域的前沿课题和研究热点，在多媒体通信、人机交互、公共安全监控、机器人导航、智能汽车等领域具有广阔应用前景。. 本项目对基于分布式麦克风阵列的多说话人跟踪理论与方法进行了深入系统研究，主要成果有：(1)基于分布式麦克风阵列的多说话人声源定位新方法，包括基于压缩感知理论的多声源定位方法、基于极限学习机的多声源定位方法等；(2)基于分布式麦克风阵列的多说话人跟踪新方法，包括基于粒子滤波和交互多模型的多说话人跟踪方法、基于无迹粒子滤波和数据关联的多说话人跟踪方法、基于时延估计的多说话人跟踪方法、基于分布式粒子滤波和迭代协方差交叉的说话人跟踪方法、基于自适应交互多模型与粒子滤波的说话人跟踪方法、基于交互多模型的信息加权一致性方法等；(3)分布式麦克风阵列校准新方法，包括基于波达方向信息的三维节点几何校准方法、基于分布式阻尼牛顿优化的几何校准方法、基于网络牛顿分布优化的几何校准方法、声学收发阵列解析式几何校准方法、约束总体最小二乘校准方法，基于一致性策略的频率响应校准方法、基于频谱平坦度的频率响应校准方法、基于多通道维纳滤波的频域响应校准方法、自适应频率响应校准方法、主动式采样率校准方法等；(4)基于分布式麦克风阵列的语音增强新方法，包括基于BiLSTM网络的多目标多通道语音增强方法、基于简单循环单元网络的语音增强方法、基于粒子滤波和动态规划的主旋律提取方法、基于伪二维谱的复调音乐多音高估计方法、基于音符跟踪和音高映射的主旋律提取方法、结合改进欧几里得算法和动态规划的主旋律提取方法、基于G.722.1的分布式语音编码方法等；(5)建立了分布式麦克风阵列多说话人定位与跟踪系统。. 本项目有效地提高了多说话人定位与跟踪系统的性能，有力地推动了麦克风阵列语音处理技术的快速发展。