面向图像超分辨率与修复的深度学习方法研究

The image super-resolution and inpainting technologies, which can reconstruct high-quality clear image from degraded and damaged image, play a key role in culture, public security, medical science and military fields. Due to the limited representation and learning ability of the shallow method, the quality of reconstructed images is severely restricted. This project focuses on investigating the deep learning methodology for image super-resolution and inpainting. Combining the deep learning principle with the sparse representation theory, firstly, we will propose a new sparse residual deep neural networks (SRDNN) and training strategy to deal with the high information redundancy, missed prior knowledge and low learning efficiency in the classical deep learning methods. Secondly, we will present a new sparse residual generative adversarial networks (SRGAN) and learning method to overcome the lack of training simples in some special application. Finally, we will propose some new image super-resolution and inpaining methods via SRDNN and SRGAN to synchronously reconstruct fine details and complete the missed content. The achievements can not only improve the diversity, robustness and effciency of feature representation in deep neural networks, but also expand training data without distortion. The proposed SRDNN and SRGAN will enrich and develop the deep learning theory. Moreover, the achievements will expand the application areas of deep learning theory and provide technical support for some important fields, such as the digital inpainting and high definition display of painted murals in the national key cultural relics - Dunhuang grottoes, and criminal investigation. This project has important theoretical significance and practical application values.

从低分辨率损毁图像中重建高分辨率无损图像，是文化、公共安全、医学和军事等领域必需的技术。由于浅层学习方法对复杂图像表示和学习能力的不足，严重阻碍了图像超分辨率与修复方法性能的提升。本项目将深入研究面向图像超分辨率与修复的深度学习方法,主要包括:构造稀疏残差深度网络模型与学习策略，解决深度网络中信息冗余度高、先验知识融入困难和训练效率低等问题；构造稀疏残差生成式对抗深度网络模型与学习策略，解决训练数据匮乏的问题；提出图像超分辨率与修复新方法，精确重构同步修复低分辨率破损图像的细节信息和缺失内容。本项目的研究成果不仅能够提高深度网络特征表达多样性、鲁棒性和学习实时性，突破无失真扩张训练数据的技术瓶颈，丰富深度学习理论；还将拓展深度学习应用领域，为国家重点文物——敦煌石窟壁画的数字化修复与高清展示，以及部分刑侦分析与判别等紧迫需求提供技术保障，具有重要的理论意义和应用价值。

利用低分辨率损毁图像重建获得高分辨率无损图像，是文化、公共安全、医学和军事等领域必需的技术，本项目针对现有基于深度学习方法的图像超分辨率和修复应用中存在的专家经验和网络学习融合不充分、深度网络中信息冗余度高，以及图像超分辨率和修复能力受限等问题，深入研究基于深度学习的图像超分辨率与修复方法。首先，引入先验信息表示与融合方法，设计了多种稀疏深度网络模型及其训练策略，提升深度网络压缩效率并最大限度保持网络性能；其次，结合多尺度特性、上下文信息复用原理以及人类注意力机制，充分利用低分辨率破损图像自身有效信息，构建了多种生成式对抗深度网络模型及其训练策略，提升深度网络特征表达多样性和鲁棒性；最后，基于上述新模型架构与训练策略深入分析低分辨率和高分辨率图像之间、破损图像与完整图像之间的映射关系，构建低分辨率破损图像以及高分辨率完整图像复杂结构间的多类回归关系模型，提出了多种基于深度学习的图像超分辨率与图像修复新方法，精确重构低分辨率图像缺失的细节信息，提高了图像超分辨率结果的准确性，还可以精准补全破损图像，使得图像修复结果结构清晰、纹理自然。本项目取得了一系列有意义的研究成果，丰富了稀疏深度网络相关理论，拓展了生成对抗深度网络相关技术，促进了基于深度学习的图像超分辨率和图像修复技术发展，可推广应用于敦煌石窟壁画的数字化修复与高清展示等应用领域，完成了本项目预定的科学研究和人才培养目标。