基于高光谱成像技术的人类组织血氧遥测与情感识别

Accurate detection of human emotion is a key initial step to the smooth human-computer-interaction. Body signals (face expression, voice, and gesture) have been employed as emotion-feature-carry signals for long. However, these signals can be hidden by will in some applications such as inspection for the anti-crime purpose.Physiological signals, such as heart beat rate, skin conductance, produced by automatic nerve systems, being not controlled by will, are more reliable emotion-feature-carry signals, however these signals are all measured in a contact way. This project employs hyperspectral imaging technique in detecting human emotion in a remote and noncontact way, and develops blood oxygenation map of human face being used as emotion-feature-carry signal for the first time in China. This project proposes to image human on the faces with hyperspectral imaging systems during they are in three emotional states, i.e. positive, negative, and calm. Based on the optimised absorption-scattering function, and optimised two-layer-skin model of human face, blood oxygenation map of the face is developed from hyperspectral image cube. And then the pattern of oxygenation distribution is studied in three emotional states, which aims to reveal the relation between the pattern and the specific emotion,and thus to develop oxygenation feature vector, which is fed into simplest classifier for classifying three emotional states. Based on the classification results, the proposed novel oxygenation feature vector is evaluated for effectivity in detecting human emotion, which will contribute to the fundamental theory of detecting human emotion in a remote and un-contact way.

)：对人类情感的正确识别是实现人机和谐交互的重要前提。身体信号（面部表情、语音、身体姿势）一直被作为情感识别的特征信号，然而这些信号，在诸如刑事侦查等特定应用背景下，容易被人为控制和掩饰。人类的生理信号，如心率、皮肤电导等不受人主观意识控制是更真实的情感特征信号，但这些信号都需要直接接触测量。本项目采用高光谱成像技术远程无接触地测量人类生理信号，并以此识别人类情感。项目基于光在人体组织中的漫散射原理建立面部血氧分布图，并将血氧分布作为情感特征信号，拟获取在正面、负面、正常平静情感状态下的人类面部高光谱图，并依据优化的面部光吸收-散射方程、优化的面部二层皮肤模型构建面部血氧分布图，研究面部血氧在各种情感状态下的分布模式，探索提取血氧分布特征，尝试实现对情感分类，并依据分类结果评价血氧分布这一全新的情感特征的有效性，为远程无接触识别人类情感提供理论基础。

项目按照既定的研究计划开展了细致全面的研究工作，完成了原定的研究目标。项目根据血红蛋白、皮肤的光学特性将450nm-850nm波段分为5个子波段，逐段考察了个子波段的成像优劣，定位了最优子波段[518-580]nm；将波段选择问题转化为路径选优问题，开发了PN-GA-PLS算法，用于波段选择；将识别率作为优化目标，定位了5个最优成像波长点，用于血氧实时生成；建立了219人的高光谱情感数据库，包括3-5种情感类型，为后续的理论研究打下了坚实的基础；探讨了采用面部血氧图识别负性情感的可能性，比较分析了面部各个区域在负性情感下的血氧值，发现面颊部分的血氧值较额头部分对负性情感更加敏感；研究了正性情感（高兴）和负性情感（悲伤）针对平静情感的变化特点，发现了面部血氧在这两种情感下呈现不同的分布模式，并初步得出了两种情感的分类识别率87%；提出了采用面部血氧图识别心理应激情绪的理论和方法，采用额头区域作为待研区域的方案，并提取了压力指数（stress index）作为识别的特征，系统提出了基于高光谱技术的情感识别的流程和方法。. 项目拓展性地研究了增强显示高光谱图像特定波段数据的方法，可用于后期将血氧图像与彩色图像叠加，并增强显示血氧图，达到更好的血氧显示效果；开发了多模态的非接触式情感识别方法，包括利用眼动信号识别专注状态，利用深度式视频提取呼吸信号，这些方法可与高光谱成像技术进一步融合，作为多模态非接触式情感识别地有效方法； 采用高价多元多项式构建了连续情感识别模型，构建和优化了卷积神经网络，可为后期建立高光谱情感识别算法作为支撑。