基于动态参数深度模型的图像超分辨率算法研究

Deep learning based image super-resolution methods have achieved the state-of-the-art performances. However, the existed deep learning based image super-resolution methods have failed to accurately reconstruct high-resolution (HR) images according to the specific low-resolution (LR) images adaptively. In addition, the image internal priors have been ignored and it is assumed that LR images are uniformly degenerated from the HR images. All these limitations lead to failures of performances in the practical scenario. To address these issues, this project proposes to apply the deep convolutional neural networks of dynamic parameters to jointly model the image internal / external priors and degradation processes from HR images to LR images, aiming to explore practical image super-resolution methods that are adaptively modeled for the specific input LR image. In this project, (1) a dynamic parameter generation network is designed and the input featuremaps are applied to dynamically modulate the model parameters for reconstructing the HR image; (2) the errors of restoring LR examples from the internal coarse examples are exploited to dynamically correct the reconstruction of HR images; (3) image degradation process is explicitly or implicitly estimated to dynamically adapt the mappings from LR to HR images. This project aims to dynamically generate the parameters of the models by jointly modeling the internal / external priors and the degeneration process to adaptively restore HR images according to the input images, which would greatly improve the interpretability and effectiveness of the model as well as achieve more accurate and visually more pleasing results.

基于深度学习的图像超分辨率算法能够取得当前最好的性能。然而，现有的该类算法无法根据输入低分辨率图像自适应地重建高分辨率图像，并且存在忽视图像内部先验信息及假设低分辨率图像的生成满足单一图像退化过程的局限性，算法无法在实际应用中精确重建图像。针对以上不足，本项目提出采用动态参数深度卷积网络，联合建模图像内外部先验和图像退化过程，旨在研究满足应用需要的、随着低分辨率图像输入自适应建模的图像超分辨率算法。本项目将（1）设计动态参数生成网络，采用输入特征图动态调整模型参数，自适应地重建图像；（2）计算内部样本重建低分辨率图像样本的误差信息，动态修正高分辨率图像重建结果；（3）显示或隐式估计图像退化过程信息从而动态地调整模型映射关系。本项目将根据图像内外部先验及退化过程信息动态生成模型参数，从而为低分辨率输入图像自适应重建高分辨率图像，极大地提高模型的可解释性与有效性，得到更加准确、赏心悦目的结果。

图像超分辨率是针对退化的低分辨率图像清晰准确重建高分辨率图像的技术，图像超分辨率在视频监控、军事、辅助驾驶、消费电子等多个领域都有广泛的应用前景。本项目针对现有图像超分辨率算法无法根据输入图像自适应重建高分辨率图像的局限性，探索了图像内外部先验及退化信息的高效建模方式；探索了图像退化过程对图像超分辨率任务的影响；实现了多种动态参数深度模型；针对实际应用场景，显著提升算法重建高分辨率图像的主客观评价效果。本项目提出了一种采用内部先验调整外部先验模型的策略，能够有效结合图像内外部先验、图像退化信息，提升图像超分辨率模型的效果；提出了由局部到全局的分层动态参数模型，能够有效提升图像超分辨率算法的模型容量，提升算法性能；提出了基于无监督学习，动态调节亮度信息的广义图像超分辨率算法；提出了一种基于自监督学习的模型动态自适应方法，能够克服算法对训练数据的依赖，提高算法对实际应用的适用性。. 项目组在论文发表、专利申请、原型系统构建、成果转化、人才培养、学术交流等多个方面达到了预期目标。本项目的研究意义在于提出了新颖的动态参数模型构建方法、有效的图像内部先验及退化过程建模方式，提高了图像超分辨率算法对输入图像的自适应性，有望提升图像超分辨率算法在实际应用中的实用性，推动图像处理领域及动态参数模型领域的发展