基于结构信息编码的鲁棒视频理解算法研究

With the progress of the AI technique, deep learning has a far-ranging applications in the video content analysis and understanding task. However, deep learning is vulnerable to adversarial samples. which makes the current video understanding algorithm show the weak robustness. In this project, we will mine the specific structured information within the video data, and further study the attack and defend methods for adversarial samples based on the temporal-spatial relationship. In this way, we finally aim to propose the robust video understanding algorithms. This project (1) proposes an efficient temporal-spatial relationship encoding method based on the manifold learning and neural network, in order to precisely capture the corresponding relation between pixels; (2) By mining the sparsity and propagation of video adversarial samples, we present a method for computing the video adversarial perturbations based on the group lasso model; (3) Combing with the efficient and precise temporal-spatial relationship, we propose a method to defend video adversarial samples based on the sampling. In general, the main objective of this project is to advance the theories and methods in robust video understanding via the studying of adversarial samples and structured information, which is of important theoretical and practical significance in terms of the mechanism of deep learning and video understanding.

随着技术的发展，深度学习在视频内容分析与理解中得到广泛应用，然而，该技术却非常容易受到对抗样本的攻击，使得目前的视频理解算法鲁棒性不足。本课题结合申请者过去在结构化信息方面的工作基础，在深入分析面向视频媒体数据的对抗样本应有特性的基础上，通过构建以时空关系结构为核心的攻击和防守模型，研究鲁棒的视频理解算法。本课题（1）提出了基于黎曼流形融合神经网络的高效时空关系表达算法，以期实现对视频中帧间像素对应关系的准确捕捉；（2）通过挖掘视频中对抗样本稀疏性和传播性的特点，提出了基于组稀疏模型的可传播对抗样本生成技术；（3）结合高效精准的时空关系表达，提出了基于帧间残差采样的可靠对抗样本防守技术。总体来说，本课题将从视频媒体数据的时空关系表达、对抗样本生成和防守等角度进行研究，力求实现相关理论的突破和创新，研究成果对更安全的进行视频理解以及更深层次理解深度学习技术都具有重要意义。

针对以深度学习为基础的视频内容智能分析和理解算法在面临对抗攻击时鲁棒性不足的问题，本课题结合申请者过去在结构化信息方面的工作基础，在深入分析面向视频媒体数据对抗样本应有特性的基础上，通过构建以时空关系结构为核心的攻击和防守模型，研究鲁棒的视频理解算法。 本课题包含三个研究内容，分别是：(1)提出了基于黎曼流形融合神经网络的高效时空关系表达算法，以期实现对视频中帧间像素对应关系的准确捕捉; (2)通过挖掘视频中对抗样本稀疏性和传播性的特点，提出了基于组稀疏模型的可传播对抗样本生成技术; (3)结合高效精准的时空关系表达，提出了基于帧间残差采样的可靠对抗样本防守技术。经过三年的研究，本课题共发表人工智能领域顶级期刊和会议9篇，研究成果对安全鲁棒视频理解以及更深层次理解深度学习技术具有重要意义。