基于深度学习的多尺度本质图像提取方法

Multiple factors influence the appearance of an object in an image. However, extracting the intrinsic images from the observer image can eliminate the environmental impact effectively and make the image understanding more accurately. The intrinsic images represent the inherent shape, colour and texture information of the object. In this proposal, we aim to explore a method of multi-scale intrinsic image extraction based on deep learning. . Based on the framework of multi-scale analysis and reconstruction, an image can be decomposed into multi-scale and multi-orientation. In order to extract the intrinsic information from those components, we proposed to build a series of global and local features. The global features are based on the correlation between image intensity and shape, colour, texture, respectively. The local feature is constructed based on multi-scale deep learning methods, which can extract the multi-scale colour and texture information. Based on correlation features, the global and local features are combined to extract intrinsic information from multi-scale components. Consequently, the intrinsic images are reconstructed from those extracted information with the multi-scale analysis framework. . The main contribution of the proposal is the introduction of multi-scale analysis to intrinsic image extraction with the measurement of features, where most current intrinsic image extraction methods are based on solving optimization problems for the observed image directly. At the same time, we proposed to build global and local features based on the characteristics of each scale component.

图像成像过程受多种因素影响，使得图像中物体表象也随之变化。而提取物体本身固有形状、颜色、纹理等本质信息，可有效消除环境影响，实现对图像的准确理解。本课题旨在探索一种基于深度学习的多尺度本质图像分析方法，实现在多尺度上详细分析本质信息的目标。通过多尺度分析与重构框架，实现对图像的自由分解与无损重构。为实现对各尺度分量中本质信息的准确提取，分别基于图像亮度与形状、颜色、纹理关系构建全局特征；同时，基于多尺度深度学习网络构建局部特征。并利用相关性特征结合全局与局部特征，实现对各分量的准确分析。最后，对各分量中提取的本质信息通过重构框架实现无损重构。. 主要创新点在于摒弃了现有算法在单一尺度下通过求解能量优化问题估计本质图像的思路，而提出了多尺度下利用特征评价直接分析本质图像的方法。并针对各分量特点，构建了基于相关性的全局特征以及多尺度联合的局部特征，实现对本质图像的准确提取。

图像成像过程易受环境的影响，往往使得同一场景在不同的环境中展现出截然不同的图像。图像的这一特性，使得场景分析、目标识别等应用在复杂环境中的性能急剧下降。本研究针对此问题，提出从图像中提取目标物体本质特征的方法，从而消除环境因素对目标成像的影响。其中，本质特征是指描述目标物体固有的颜色、纹理、形状等特性。若能够从图像中准确的提取出目标物体的本质特征，则可有效的消除环境的影响。本项目首先提出利用导向滤波器组对图像进行多尺度分析，进而构建了描述基于相关性的多尺度特征可以有效描述目标物体的反射特性和光照特性。基于导向滤波器组的完备性以及构建的多尺度特征，可以实现对本质图像的有效重构。其次，基于深度学习网络强大的学习能力，本项目构建了深度学习网络模型可以同时获得对反射特性和光照特性的估计结果。利用端对端的深度网络模型无需再手工设计相关特征，可通过训练自适应的获得重构方法，实现对本质图像的准确重构。再次，为了进一步提升对本质图像估计的准确率，构建了生成对抗网络估计本质图像。在该网络中，通过多尺度误差、局部注意力模型以及对抗网络的协同优化等技术，实现了对本质图像较为精准的估计。最后，本项目还整理构建了一个包含有22万张图像对的本质图像数据库，为完成对相关算法的训练和客观评测提供了数据基础。