基于EEG引导的短视频质量深度分析方法研究

The quality of short videos plays an important role in the field of video analysis and interpretation, personalized recommendation, video information dissemination and public opinion monitor. However, the state-of-the-art models of video quality assessment (VQA) can only provide one quality score which is not accurate enough for real-time applications. Therefore, this project aims to extract the features of short video, design the quality assessment measurement, and predict the perceptual quality, under the guide of the electroencephalogram (EEG) which is recorded in the subjective experiments. The detailed research contents include: (1) building the relationship between the EEG signal and the perceptual quality of short videos through analyzing the spatio-temporal characteristics of the multi-channel EEG signal, the formation mechanism of perceptual quality, and the intrinsic mapping relationship. (2) representing the features of short videos guided by EEG signal through recovering the variation of EEG signal with the sequence of short videos, and mining the perceptual features of short videos. (3) designing the perceptual quality measurement of short videos through building the deep network with ability to simulate the human visual system, and obtaining the final quality sequence in perceptual process. In summary, based on the physiological mechanisms of EEG, a new framework of short video quality assessment is constructed via combining with multi-view learning and deep learning. It can provide some new ideas for quality assessment theory, and also provide the theoretical basis and technical support for studying the perception and cognitive mechanism of the human visual system.

短视频的感知质量在视频内容的分析与解译、精准的个性化推荐、信息传播与舆情监控等领域中起着关键性作用。然而，现有视频质量评价模型只给出单一的评价分数，且评价结果不够准确。鉴于此，本项目通过分析主观评价过程中的脑电信号，以此来指导短视频的特征表示、评价测度设计和质量过程评价。具体包括：（1）短视频感知脑电信号与质量分数的关系建模，研究多通道脑电信号的时空特性、感知质量分数的形成机理，以及两者之间的隐含映射关系；（2）EEG引导的短视频特征学习与表示，研究脑电信号随短视频空时特性的变化规律，以此为引导挖掘短视频的感知质量特征；（3）社交媒体短视频质量评价测度设计，研究模拟视觉特性的深度网络，实现感知质量的过程评价。本项目基于短视频感知脑电的生理学特性，结合多视角学习和深度学习等最新机器学习方法，构建短视频质量评价的新框架，丰富和完善质量评价理论，为视觉信息的感知认知机理提供理论依据和技术支撑。

短视频质量感知是视觉认知机理的基本问题之一，具有重要的理论意义和应用价值。本项目主要研究了脑电信号引导的短视频质量感知机理，设计感知实验范式并测定感知过程中的脑电信号，构建脑电信号与视觉质量之间的关系，探索短视频质量感知过程中的生理和心理机制，将感知特性应用于客观视频质量评价。本项目在以下几个方面取得重要进展和成果：（1）研究了基于脑电信号的短视频质量感知机理。研究了短视频失真时长、空间复杂度、空域失真范围以及色饱和度与感知质量之间的关系，发现在相同失真程度下，视频失真时长越长，脑电信号峰值越大，低空间复杂度视频中的失真更易被感知，且失真范围越大感知越明显，饱和度越低，短视频中的失真越容易被感知。研究了短视频中目标的显著性与运动速度对于感知质量的影响，发现目标显著性越强视频失真越容易被感知，目标运动速度越快视频失真越容易被感知。该研究为短视频客观质量评价方法设计提供理论支撑，有效地扩充了视觉信息的感知-认知机理。（2）研究了基于脑电信号的短视频质量评价方法。构建了基于对抗训练和多被试脑电信号迁移学习的短视频质量分类方法，解决了多被试个体差异导致的模型训练中的负迁移问题；构建了基于脑电信号和时空多尺度联合神经网络的短视频质量分类方法，解决了脑电信号时空特征提取困难的问题；构建了基于注意力机制和时间卷积网络的质量分类方法，解决了不同空间和时间上脑电信号的特征混同问题。（3）研究了基于视觉感知特性的短视频质量评价方法。根据短视频的视觉暂留和记忆等机理，设计了基于元迁移学习、显著性排序、近因效应池化、帧组内外依赖等短视频质量评价模型和方法，取得较好的主客观一致性。