语音同步的高真实感三维人脸情感动画研究

In view of intelligent human-machine interaction, the research on speech synchronized emotional 3D facial animation is developed for multi-modal human-machine interaction. It is expected to achieve following goals: a speech synchronized 3D facial animation system is proposed based on the establishment of facial animation scheme driven by text and speech. This system can run in real-time and has the capabilities of producing high realistic facial animation and expressing emotion. As a result, for the problem between easy use and high reality and the difficult between computation effort and real-time, following problems will be studied thorougly, i.e., facial animation with 3D facial model, the modeling of 3D articulatory motion, the modeling of cooperation between articulators and speech. Moreover, key technologies will be established and the practice speech synchronized 3D facial animation system will be constructed to provide a concrete foundation for application.

本项目以智能人机交互为研究对象，从多模态人机交互问题入手，系统开展语音同步的三维人脸情感动画研究。希望通过该项研究，达到如下目的：设计并实现文本和语音多种输入驱动的三维人脸动画方案，实际构建出语音同步的、具有高自然度人脸动画合成功能和情感表达功能的实时三维人脸动画系统。针对系统实现过程中所面临的易用性与高自然度之间、算法的计算复杂度和实时性之间可能存在的矛盾和难题，拟从系统的角度，对基于人脸三维模型的人脸动画合成，三维发音器官运动建模，发音器官和语音的协同关系建模等诸问题进行深入研究，确立与之相关的关键技术并实际构建出以这些关键技术为基本构成元素的语音同步的三维人脸动画系统，为研究走向实用化奠定基础。

本项目以智能人机交互为研究对象，从多模态人机交互问题入手，系统开展语音同步的三维人脸情感动画研究。通过该项研究，达到了如下目的：设计并实现了文本和语音多种输入驱动的三维人脸动画方案，实际构建出了语音同步的、具有高自然度人脸动画合成功能和情感表达功能的实时三维人脸动画系统。针对系统实现过程中所面临的易用性与高自然度之间存在的矛盾和难题，从系统的角度，对基于人脸三维模型的人脸动画合成，三维发音器官运动建模，发音器官和语音的协同关系建模等诸问题进行了深入研究，确立了与之相关的关键技术并实际构建出了以这些关键技术为基本构成元素的语音同步的三维人脸动画系统。已在IEEE Transactions on Cybernetics、IEEE Transactions on Circuits and Systems for Video Technology、Science China - Information Sciences、IEEE Signal Processing Letters、Multimedia Tools and Applications、IEEE International Conference on Multimedia Expo、IEEE International Conference on Acoustic, Speech and Signal Processing、IEEE International Conference on Image Processing、International Conference on Multimedia Modeling等国际权威期刊和会议上发表科技论文47篇。授权中国专利2项。