基于多视角学习的情感分析理论与方法研究

As one joint research field of cognitive psychology, neurophysiology, behavior science and computer science, affect analysis plays an important role in various application areas such as intelligent surveillance, robotics, human-machine interaction, emotional cues analysis in criminal investigation, video indexing. This project will explore and study a novel method for multiview (visual, audio, and other physiological signals) affective signals cognition, affective feature extraction, and affective status analysis. Firstly, we will propose a biologically inspired multiview model for multiview affective signals perception, feature presentation, and affective status description. Secondly, we will propose a novel affect feature extraction method which incorporates sparse coding and Hessian regularization. Thirdly, we will propose a novel multiview Hessian regularization framework which can effectively explore the complementary properties of different features from different views and then boost the performance of affective status analysis significantly. Finally, we will construct an online affect analysis system to evaluate the affect status in real-time. The proposed study of multiview affect analysis which integrates cognitive psychology and neurophysiology will break through traditional affect analysis framework and develop a fresh view to interpret physiological and cognitive mechanisms of emotional behaviors.

作为认知心理学、神经生理学、行为科学以及计算机科学的交叉研究内容之一，情感分析在智能监控、机器人、人机交互、刑侦情感线索分析、视频检索等多个领域有着重要的应用价值。本项目拟探讨和研究多视角（视觉、听觉、其他生理信号）情感信号感知、情感特征表示以及情感状态分析方法。首先借鉴认知心理学和神经生理学机制，提出一种多视角分析模型，用于进行多视角情感信号感知、特征表示和情感状态描述；其次，提出基于Hessian正则化的稀疏编码算法，用于情感特征提取；第三，提出一种多视角Hessian正则化模型，有效的利用不同视角特征的互补作用进行情感状态估计；最后，构建情感分析系统，对情感状态进行实时分析。本项目将突破传统的情感分析框架，结合认知心理学和神经生理学进行多视角情感分析，为阐明情感行为的生理机制，揭示情感行为的认知规律提供一条新的途径。

本项目以情感分析为出发点，分析了情感信号的多视角特征，提出了多视角情感分析方法，为情感行为分析提供了解决途径；进而以此为基础，本项目对多视角学习、稀疏编码及流形正则化理论进行了深入研究，提出了多视角Hessian正则化的稀疏编码算法、多视角Hessian正则化的逻辑回归算法、Co-training算法、p-Laplacian正则化的稀疏编码算法及典型相关分析算法等多种多视角学习、稀疏编码和流形正则化算法，并将这些算法应用于人脸分析、图像识别/标注、动作识别、社会行为分析等多个多媒体计算应用问题，获得了很好的应用效果。项目发表各类期刊论文13篇（其中SCI/EI期刊12篇）、会议论文14篇，培养硕士毕业生3名，获山东省高等学校优秀科研成果奖三等奖1项。本项目研究丰富了多视角学习、稀疏学习及流形学习理论，为情感分析、图像识别及人体动作分析等多媒体计算提供了有效的解决途径。