基于多维度数据的言语真实性辨识技术研究

The study on authenticity identification of spoken language can meets the great demands of our national security. However, current authenticity identification study has poor identification ability due to their invasive and single-dimension natures. In this study, the state reaction patterns of our brain in the related tasks of spoken language deception will be studied using fMRI technology at first. Secondly, the normalized phase features of full band will be designed based on fundamental frequency synchronization, and next, together with other complementary features (like amplitude, prosodic, etc.), the relations between ordinary liars and professionals will be further modeled by a specially designed deep neural network, and a two-tire transfer learning algorithm will be proposed to realize the adaptive training on the authenticity identification model for professionals. Thirdly, the relations between temporal series will be studied using our proposed deep neural networks based on generative temporal model by fusing attention mechanism and long-term memory mechanism, in addition, discriminative representations of eye movement features, facial expression features and micro expression features will be further extracted. Fourth, multi-modal feature fusion method based on Hilbert-Schmidt Independence Criterion will be proposed based on new designed network structure. Finally, a deceit database will be constructed, and a spoken language authenticity identification system based on multidimensional features will be designed and validated. The ideas on the proposed feature extraction method, network model design method and multi-modal fusion method in this study are of practical significance to the development of the research in the fields of cognitive computing, speech signal processing, pattern recognition and so on.

言语真实性辨识研究符合国家安全重大需求。现有的方法基于接触式、单维度，辨识性较差。本项目拟使用fMRI考察大脑在言语欺骗相关任务中的状态性反应模式；设计基于基频同步的全频带归一化相位特征，并融合与相位信息有高度互补性的基于振幅、韵律等特征，设计专门的深度神经网络，对普通说谎者与专业人员的关系进行建模，提出双层迁移学习算法实现针对专业人员的真伪辨别模型的自适应训练；提出结合注意力机制和长时记忆的生成时序模型的深层神经网络模型，有效学习时间序列之间的关系，并构建出具有高辨别性的眼动特征，面部表情和微表情特征表示；通过设计新的网络结构，建模基于希尔伯特-施密特独立标准的多模态特征融合方法；搭建测谎数据库，实现一套基于多维度数据的言语真实性辨识系统，并验证所提方法的有效性。提出的特征提取、网络模型设计、多模态融合方法的新思路对认知计算、语音信号处理、模式识别等领域的学科发展具有借鉴意义。

言语真实性辨识研究符合国家安全重大需求。现有的方法基于接触式、单维度，辨识性较差。本研究以解决基于多维度数据的言语真实性辨识中存在的关键科学问题和技术难题为出发点，开展了基于视频微弱信号的言语真实性特征表示、基于希尔伯特-施密特独立标准的多模态内容关联及多特征融合、视频数据标注质量提升、基于视频的多模态特征提取等方面的研究。提出了基于三维人脸先验知识的模糊人脸信息提取方法、3D脸部结构先验的人脸超分辨算法、暗光图像中边缘细节信息恢复方法、基于空间异变循环神经网络的微弱场景信息提取方法、针对有污染相对比较的鲁棒有序嵌入、基于SplitLBI的平局可感知偏序学习、面向负迁移的任务-特征协同学习、基于视频的多模态特征提取方法等一系列方法。本项目搭建了测谎语料库，实现一套基于多维度数据的言语真实性辨识原型系统，并验证所提方法的有效性。本研究中所提出的新理论新方法具有巨大的潜在应用前景。