基于判别学习的图像盲超分辨和盲复原模型及算法研究

Image super-resolution and blind restoration are two fundamental yet active problems in image processing community. Most existing single image super-resolution (SISR) methods assume that the blur kernel in the down-sampling model is known, but in real world scenario the blur kernel generally is unknown and should also be estimated from the low-resolution image. Most existing blind restoration (e.g., blind deconvolution) methods are based on the explicit modeling strategy on image priors. However, recent studies on image denoising and deblurring indicate that, task-driven discriminative learning approaches can obtain better models for image priors and achieve superior reconstruction results. .. To address the above issues, in this project, studies will be conducted on blind SISR and blind restoration from three aspects: (1) we will analyze the intrinsic connection between blind SISR and blind restoration, and then build one unified framework for these two problems; (2) Discriminative learning of recurrent deep model will be established for learning both the confidence term and the regularization term in a data and task driven manner; (3) Robust non-blind SISR and deblurring methods will be developed to improve the algorithm robustness against inaccurate blur kernel estimation. .. To sum up, this project can not only suggests a generalized unified model for blind SISR and blind restoration, but also extends the discriminative learning models to learn the confidence term. Moreover, for the first time, systematic studies on the robustness against inaccurate kernel estimation will also be conducted in this project. All these studies can deliver valuable insights and new tools for both academic research and real word applications. In previous work, we have developed a series of state-of-the-art methods for image denoising, super-resolution, and restoration. Our preliminary results on blind restoration and discriminative image modeling also have validated the effectiveness and feasibility of the proposed project. We strongly believe that the project will lead to significant and impactful research outputs.

图像超分辨和盲复原是图像处理领域的基础性问题。目前的超分辨方法主要建立在下采样模型已知的基础上，而实际上下采样模型中的模糊核往往是未知的。主流的盲复原方法大多仍基于现有图像先验模型，而近年来的图像去噪和去模糊研究发现，基于任务驱动的判别学习方法往往能得到更好的图像先验模型和重建效果。因此本项目将：(1) 分析图像盲超分辨和盲复原的内在机制，将其纳入到一个统一的框架；(2) 采用数据和任务驱动方式建立判别学习模型，实现对模型中正则项和信度项的建模；(3) 研究鲁棒的超分辨和复原算法，在模糊核估计不准确时改善重建算法的稳健性。本项目不仅可为盲超分辨和盲复原建立通用模型，还将判别学习推广至图像信度项学习，并系统研究算法的稳健性问题，对相关的学术研究和实际应用中都有重要的指导意义。项目组在去噪、超分辨与复原等方向具备良好的研究基础，针对图像盲复原和判别先验学习开展了前期研究，可为项目开展提供保障。

图像超分辨和盲复原是图像处理领域的基础性问题。目前的超分辨方法主要建立在下采样模型已知的基础上，而实际应用中的下采样模型中的模糊核往往是未知的。主流的盲复原方法大多仍基于现有图像先验模型，而近年来的图像去噪和去模糊研究发现，基于任务驱动的判别学习方法往往能得到更好的图像先验模型和重建效果。因此本项目将分析图像盲超分辨和盲复原的内在机制，将其纳入到一个统一的框架。进而，采用数据和任务驱动方式建立判别学习模型，实现对模型中正则项和信度项的建模。最后，研究鲁棒的超分辨和复原算法，在模糊核估计不准确时改善重建算法的稳健性，最终基于深度卷积网络建立图像超分辨和图像复原的通用模型。..围绕上述研究内容，项目针对图像去噪、图像超分辨和图像去模糊等典型任务和通用图像复原模型，提出了一系列代表性的深度网络模型。针对图像去噪，在国际上较早开展了基于深度卷积网络的图像去噪研究，提出了DnCNN、FFDNet、MWCNN和CBDNet等模型。DnCNN自2017年发表以来谷歌学术引用超过2600余次，并被官方集成至MATLAB 2017b的Image Processing Toolbox。FFDNet和CBDNet是国际上较早的灵活性去噪和真实相机图像盲去噪方面的工作。针对图像超分辨，在国际上首次考虑了退化模型的多样性问题，提出了面向多退化模型的灵活性解决方案，并结合退化模型参数估计将其进一步应用于盲图像超分辨。针对图像去模糊中模糊核估计不准确问题，提出了一种退化模型和正则化参数的联合学习模型。在国际上首次利用深度卷积网络在盲去卷积任务上取得了优于传统方法的复原性能。针对通用图像复原，通过将深度去噪模型和优化算法相结合，可为图像去噪、去模糊和超分辨等任务提供一个单一的通用模型，产生了较大的学术影响。..在项目支持下，发表IEEE T-PAMI、T-IP、IEEE TNNLS等国际期刊及CVPR/ICCV/NeurIPS/ECCV等国际会议论文50余篇，其中IEEE Trans.论文15余篇，CCF A类会议20余篇。参加CCF A/B类会议11人次，邀请国外学者讲学4次，博士生海外访学4人次。培养博士生6名，硕士生10名。