恶劣条件下视频联合增强与理解研究

Video reconstruction and understanding technology is bringing a series of revolutionary changes in the daily life and industrial production. However, due to the limitations in pixel accuracy, shooting environment, and network transmission, existing video reconstruction and understanding technologies are faced with a number of challenges: data diversity, reliance on manual labeling, etc. In bad weather conditions, the shortcomings of the existing technology are more obvious: video capture and analysis in heavily bad weather is difficult; existing analysis and understanding of technology is difficult to deal with serious video degradation; the relevance and complementary of video enhancement and understanding have not been fully studied. Thus, it is urgent and necessary to take a unified view to study video enhancement and understanding problem in bad weather conditions. The project will focus on designing and constructing a unified video enhancement and understanding framework to integrate video enhancement and understanding, to handle the challenges in heavily bad weather conditions from several aspects: We make use of the understanding information in videos to provide adaptive priors for video restorations; we exploit the video understanding features such as motion estimation and regional consistency to improve the image restoration performance under bad weather conditions; we utilize skeleton node information to deal with the video understanding analysis in bad weather conditions; we build a unified framework for video enhancement and understanding, and use cloud databases and large data stored on the Internet for online compensation. These works are important to support video enhancements and understand as well as the subsequent applications, to promote the development of related fields.

视频重建与理解技术正在为人们的日常生活和工业生产等方面带来革命性变化。但是受到拍摄设备成像精度和拍摄环境等方面限制，现有视频重建与理解技术正面临诸项挑战：数据多样性、依赖人工标定等。在恶劣条件下，现有技术的缺点更加明显：恶劣条件下的视频采集和分析十分困难；现有技术难以应对严重的视频降质；视频增强与理解之间的关联性和互补性尚未被充分研究。亟待使用统一的框架对恶劣条件下的视频增强与理解问题进行研究。本项目着眼于统一视频增强与理解框架的设计与实现，整合两类任务，以应对恶劣条件下相关问题的挑战：利用视频理解信息，提供自适应先验，约束视频复原逆问题求解；利用区域一致性等视频理解特征，提升恶劣条件下的视频复原性能；利用骨架节点信息，应对恶劣条件下的视频理解分析；构建视频增强与理解联合统一框架，并使用云数据库进行在线补偿。这些工作对支撑相关应用，推动和促进相关领域的发展都具有重要意义。

随着多媒体技术的应用环境日趋广泛，传统的视频增强和视频理解方法面临在恶劣条件下的可用性不足的问题，而使用统一的框架对恶劣条件下的视频增强理解问题进行研究，并使用统一视频增强理解框架的设计与实现将视频增强任务进行整合，能够应对恶劣条件下的诸多挑战。本项目重点研究了理解驱动的结构化稀疏表示视频重建、基于视频理解特征的恶劣条件视频复原、恶劣条件视频行为增强理解以及基于海量数据的视频增强理解统一框架。在核心算法的研究上，以恶劣条件下的视频增强与理解为支撑点，以结合视频理解的增强重建和增强重建后的视频理解为基础，以研究两类任务及其特征之间的内在关联为核心，构建视频重建与理解的统一框架，立足于对恶劣条件下获取的视频进行信号增强与语义理解。本项目着力从理论探索、算法设计和应用框架三个方面进行研究，为视频增强与理解方法在恶劣条件下的应用提供理论依据和技术指导，对基于视频增强与视频理解的相关实践有十分重要的科学意义和实用价值。.依托项目共发表学术论文76篇，其中SCI期刊论文35篇，CCF A类论文39篇，人工智能领域国际顶级期刊IEEE TPAMI 5篇，图像处理领域国际顶级期刊IEEE TIP 14篇，计算机视觉领域国际顶级会议CVPR/ICCV/ECCV共计11篇。申请国家发明专利74项，其中授权发明专利38项。