基于跨层次生成对抗网络的移动端图像超分辨率研究

Generative adversarial networks (GANs) are a class of generative models for learning probability distributions of data by formulating a minimax game between a generator and a discriminator. It has been applied in image super-resolution and achieved the progressive results. However, when enlarging the compressed images on mobile terminals, this model cannot fast and stably generate high-resolution textures and details. The reasons include the diversity loss of generated features, the imbalance for optimizing the generator and the discriminator, the inaccurate objective functions and the instability of training performance. To solve these problems, this project focuses on how to design and train GANs for producing the high-quality super-resolution reconstruction. We propose some approaches to improve the generation performance, which include the dense layer connection of cross-level features, the combination with multi-scale shearlet transform, the improved loss functions. These methods are expected to obtain the accurate mapping of feature spaces at different resolution levels, and generate a high-resolution image with natural and reliable textures. In order to explore the ways to enhance the training stability and accelerate the convergence rate, this project adopts the subgradient method and the weight normalization technique to perform the training optimization. Moreover, only the principal weights are normalized according to the spectral properties of a high-dimensional weight space, which are deduced by using random matrix theory. The proposed super-resolution algorithms would be integrated and tested on the terminal devices. This study has important sense to clarify the adversarial mechanism of learning process and reveal the generation rule of image super-resolution.

生成对抗网络是一种通过内部对抗机制进行学习的概率生成模型，在图像超分辨率领域已取得显著进展。但对移动端压缩图像放大时，还不能快速稳定生成高分辨率的纹理细节。这是由于当前研究还未解决该类模型存在的生成特征多样性不足与失真、生成和判别网络优化过程的不同步平衡、目标函数表达不准确以及训练性能不稳定的问题。为解决这些关键问题，本项目研究如何设计与训练对抗生成网络才能实现高质量超分辨率生成，提出对网络的高密集跨特征层次连接，并与多尺度Shearlet变换结合的建模方法，设计合理的目标函数，可望得到不同分辨率特征之间的准确映射关系，生成失真少纹理丰富的高分辨率图像。本项目在训练时使用次梯度优化方法，利用随机矩阵理论分析高维权重空间的谱性质，确定主成分权重并进行归一化处理，探索提高训练稳定性和收敛速度的有效途径。本研究算法在移动端设备进行原型验证，对阐明对抗学习机制，揭示图像超分辨率生成规律有重要意义。

本项目重点针对移动端压缩图像的超分辨率生成任务，深入研究了生成对抗网络的设计、训练、优化与实际部署。为了快速稳定得到高质量的图像超分辨率重建结果，本项目对生成对抗网络的结构设计与建模方法进行研究，提出了基于注意力机制的多尺度网络模型，结合小波变换的特征动态融合与调制模型，以及基于高阶语义关系的模型，这些模型可以学习不同分辨率特征之间的准确映射关系，输出高保真的超分辨率结果。在对训练过程研究时，提出了基于预训练模型的有效目标函数，设计了精确的概率分布距离测度，增强了超分辨率图像的视觉质量，改进了判别器的结构及训练方式，使用一致优化理论同步更新梯度，有效提高了训练的稳定性和收敛速度。优化网络模型时，利用随机矩阵理论分析高维权重的谱性质，确定主成分权重并进行归一化处理，加速收敛并提升了模型的泛化能力。对潜在空间进行正交分解，实现可控的语义解耦，完成图像超分辨率生成系统的移动端快速轻量化部署。本项目阐明了对抗学习机制，突破了图像超分辨率技术在移动端部署的限制，对全面提升生成对抗网络的设计、训练与优化的理论与应用水平具有重要的科学意义。