甚低分辨率目标图像的非参数盲超分模型与快速算法研究

Deep learning-based single image super-resolution (SR), as a frontier low-level vision problem, has earned extensive attractions among the internatonal image processing and computer vision commmunity. In this proposal, we aim at overcoming the common disadvanges in existing deep learning-inspired SR methods, such as the non-blind parametric assumption, non-adaption to various blur-kernels, and so on, through developing a couple of novel critical ideas, including the discriminative sparsity regularization for fast and exact nonparametric blur-kernel estimation and semi-coupled learning of high- and low-resolution convolutional filters in the deep SR neural network. With the core contributions above, the nonparametric blind super-resolution for extremely low-resolution target images can be formulated, including the reconstruction models and numerical schemes. In detail, three specific contributions are to be made if this proposal is successfully funded. First of all, a new computable image sparsity prior is to be studied which can discriminate between the blurred image and its counterpart clear image, based on which a fast nonparametric blind deconvolution algorithm is derived via combing the fast Fourier transform (FFT) and augmented Lagrangian scheme. On the second, the semi-coupled learning mechanism of high- and low-resolution convolutional filters is to be studied which will guide the subsequent training of non-blind deep SR convolutional network, which is designed adaptive to nonparametric blur-kernels. Finally, in the blind SR scenario two new methods for nonparametric blur-kernel estimation will be studied by making use of the convolutional consistency constraint as well as the Gaussian conditional random fields, which then finally lead to high-qulity blind SR reconstruction for extremely low-res target images.

基于深度学习的图像超分辨率(Super Resolution; SR)是国际上备受关注的前沿性课题。本项目将以SR深度学习方法的参数化非盲假设等共性缺陷为突破口，以鉴别性稀疏先验约束的非参数模糊核精确快速估计、以及SR深度神经网络的卷积滤波器半耦合学习为核心科学问题，系统研究甚低分辨率目标图像的非参数盲超分模型与快速算法。具体而言，本项目主要包括以下研究内容：首先，探讨具有模糊-清晰图像鉴别能力的稀疏正则化可计算先验模型，以此设计基于快速傅立叶变换和增强拉格朗日迭代的非参数盲去卷积方法；其次，通过挖掘高、低分辨率卷积滤波器之间的半耦合学习机制，探讨非参数模糊核自适应的图像非盲SR深度卷积模型新型架构及其训练方法；最后，基于高斯条件随机场与卷积一致性约束探讨图像盲SR非参数模糊核快速估计方法，最终实现甚低分辨率目标图像的高质量超分重建。

本项目针对视频监控等场景中不受限图像非参盲去模糊以及盲超分辨这两个紧密相关的计算成像问题，先后从先验模型探讨和数值算法设计两个方面开展了系列的创新研究。具体而言，在基于双重鉴别先验的图像恢复非参模糊核估计模型与算法这条主线上，项目先后提出基于加权超拉普拉斯双重鉴别先验、以及基于redescending potential regularization（RPR）的非参模糊核估计方法，且在自然图像盲去模糊和盲超分辨问题尤其是前者有效验证了上述方法的有效性和鲁棒性，为进一步探讨诸如人脸等的超低清目标重构奠定了模型基础；在基于生成对抗网的图像恢复端到端有监督学习模型与算法这条主线上，项目先后提出了基于Wasserstein generative adversarial network （WGAN）、以及Relativistic generative adversarial network（RaGAN）的有监督图像恢复方法，探讨了WGAN和RaGAN在不受限图像非参盲恢复问题上的学习潜能，为项目转而深入探讨自监督和无监督方法奠定了实践基础；在自监督和无监督先验提升的图像非参恢复统一模型与算法这条主线上，项目在RPR方法框架下先后提出了引入自监督先验Deep Image Prior（DIP）、以及无监督先验progressive growing of GANs（PGGAN）的图像非参恢复统一优化泛函，最终在自然图像和人脸图像上分别验证了结合DIP和PGGAN的超低清图像非参盲超分RPR正则方法的特别有效性。本项目已公开发表或录用SCI检索期刊论文10篇、EI检索会议论文8篇、中文核心期刊论文3篇，授权发明专利3项。