融合视频人脸及唇动密码特性的身份鉴定关键技术研究

This project presents an identity verification approach by fusion of the video face and lip-password, which combines the merits with respect to the strong security of the multi-biometrics and the privacy of password information. This type of approach holds multiple protections to the Identity Verification System (IVS) provided that the individual is verified by the video face, the underlying behavior characteristics and the specific password information within the lip motion simultaneously. In this project, we shall concentrate on studying the sequential feature extraction and model establishment for both of the video face and lip-password modalities, which will be very useful to discriminatively survey the video face information mining, the lip motion behaveior representation and password characteristic analysis. Meanwhile, according to the respective characteristics within these two modalities, we will synchronously investigate the the basic theories and effective approaches of their fusion in terms of the model analysis, algorithm implementation and practical application. Consequently, the corresponding human vision and machine learning based identity verification algorithms, featuring on improving all the aspects of identity identification system,can be established. After in-depth studies of the aforementioned theories, this project will mainly focus on investigating the following four key issues: 1) The rectification of the practical misalignment within the multimodal video sequence; 2) The study of the diverse model representation and its corresponding theory within the video face modality; 3) The analysis of the lip-password sequence and its discriminative model representation theory; 4) The optimal theory and methodology within the multimodal fusion based identity verification strategies.

本项目提出融合视频人脸及唇动密码特性的身份鉴定技术，该技术结合了多生物特征较强的防伪能力以及密码信息的保密性，要求对身份认证需同时囊括视频人脸信息、唇动行为特性以及特定密码口型序列信息的匹配，以起到对身份鉴定的多重保护作用。本项目以视频人脸和唇动密码序列的特征提取以及模型建立为核心，旨在进行判别性的视频人脸信息挖掘、唇动行为表示以及密码特性分析；同时，根据以上两种模态的各自特性，拟在模型分析、算法实验和应用研究三个层面探讨并研究融合这两种模态的基本理论和方法，进而建立基于人类视觉识别和机器学习识别的融合算法用于提升身份鉴定系统的各方面性能。 在深入理论研究的基础上,本项目着重研究以下四方面内容: 1)多模态误对齐下的视频序列对准方法；2)多样化视频人脸表达模型及其理论；3)唇动密码序列分析及其判别性模型表示理论；4)身份鉴定中的融合策略及其最优化理论和方法。

项目针对视频人脸和唇动密码特性的融合，利用计算机视觉和模式识别的基本理论，在判别性特征提取、唇动分割、唇动检测，唇动建模、人脸姿态估计以及融合方法等方面进行了深入的研究；同时，根据以上两种模态的各自特性，在模型分析、算法实验和应用研究三个层面探讨并研究融合这两种模态的基本理论和方法，进而建立基于人类视觉识别和机器学习识别的融合算法用于提升身份鉴定系统的各方面性能。通过本项目的研究，目前取得的主要成果有以下几点：1）创新的提出了一种基于Multi-Boosted HMMs的数字唇动密码身份鉴定方法；2）提出一种基于样本隐马尔科夫模型的唇读识别方法；3）提出一种结合人脸和数字唇动密码的分层身份验证方法；4）提出了一种基于方向性区域光流直方图和低秩相关度挖掘的唇动状态分析分析方法；5）提出了一种基于Garbor子空间相关度和自我调节谱分类方法的无监督面部姿态分组方法；6）提出了一种基于深度关联神经网络的音视频融合方法；7）提出了几种有关项目关键研究技术扩展性研究的解决办法；已发表SCI论文（10篇），国内核心期刊3篇，国际EI会议论文（6篇)，本项目合作培养硕士生 5 名（1名已毕业）.