图像情感元素计算

We human beings have the most complex emotions, yet we know so little about the secrets of human emotions, especially their carriers. From what we can see, how does it carry the emotions that we want to express. Through this project, we want to investigate the visual representation and decomposition of human emotion, and hence to find the potential visual elements of emotions. By combining the cutting-edge techniques from computer vision, statistics, and the recent results from cognitive psychology, we plan to use deep learning methods for the extraction of the sentimental elements from large-scale digital images. We believe such elements are the main carrier as well as the trigger for certain human emotions. We first try to verify the existence and test the measurability of such elements. Then, we prove the rightness and plausibility of these elements by quantifying and transferring them into different domains. After that, we will use the discovered sentimental elements to reveal the mechanism of how emotions are activated by certain visual stimuli. We also investigate and quantify the influences of the subjective factors to the final emotion representation. A unified computational model of human emotion will be constructed based on both the objective sentimental elements and the subjective factors. The ultimate goal of this project is to build a self-complete theoretical and technical system which explains the generation of human emotions given certain visual stimuli and to apply such system to solve real-world problems such as emotion classification, sentiment retrieval, and emotion prediction.

面对具有最复杂情感的人类，我们又了解多少人类情感及情感的载体。从所见之中，可知又是如何承载我们所要表达的情感的？我们想通过这个项目，求解人类情感的视觉表达和分解，从而找到视觉情感元素。项目立题的目标是希望借助计算机视觉、统计学和认知心理学的研究成果，通过深度学习手段，探索图像中是否存在承载、诱发观者情感的视觉元素，验证情感视觉元素存在性与可度量性。通过量化和迁移，证明其正确性与合理性，从而侧面揭示视觉刺激诱发特定情感的机理，并试图量化确定人类主观因素对情感表达的影响，建立人类受视觉刺激产生情感的全计算模型。项目的最终目的是构建一个完整的视觉刺激诱发特定情感的理论体系和模型，并将其用于解决情感分类、情感检索、情感预测等实际问题。

针对图像中情感视觉元素的不确定性和复杂性问题，解析情感元素特征的表示方法及作用机理，建立了基于深度特征提取及分析的的全计算模型，并将情感元素计算原理应用到多个实际视觉任务中。本项目从大规模数据集分析挖掘方法出发，探索情感元素的存在性和可表示性，建立了多层次情感元素特征表示，在情感元素量化的基础上结合深度学习、统计学以及认知心理学的研究成果，解析视觉刺激诱发特定情感的机理，考虑人类主观因素对情感表达的影响，建立情感元素与视觉媒体之间的量化关联，实现以人为中心的情感认知相关联的图像视频分析、动作识别与生成，建立了图像中情感元素感知模型及生成模型的理论与方法，克服了以往计算机视觉任务中情感影响因素的缺失问题，实现视觉诱发情感在理论层面的深化理解和算法模型层面的实际应用。