基于雨水物理特性的单图雨水条纹去除方法研究

In many computer vision methods, the input images are usually assumed to be clear for processing and scene understanding. However, rain streaks in the image captured in rain conditions tend to obstruct and blur the underlying scene, thereby degrading the accuracy of outdoor vision systems such as video surveillance and self-driving cars. This project has the two observations on rain physical property: (i) In nature, the occlusion is caused not only by the rain streaks but also by the fog that comes with the rain. (ii) The scene visibility spatially varies in the image space, since objects closer to the capture devices (e.g., the camera) are affected mainly by the rain streaks, while objects far away are affected more heavily by the fog. Existing methods and datasets for single-image rain streak removal, however, ignore these two rain physical properties, thereby limiting the rain removal efficiency on real rainy photos. To address this issue, this proposal will develop a rain physical property and semantic information based framework for single image rain streak removal. It has the following three problems in details: 1) rain physical property-based single-image rain streak removal; 2) saliency-aware non-rain background scene detail enhancement; 3) noise density-aware image denoising. The study of this project shall provide new research insights and novel solutions for single-image rain streak removal and expand the application of this technology in practice.

目前计算机视觉方法通常假设图像处理和场景理解的输入图像是清晰干净的。 然而雨水天气下采集的图像中的雨水条纹往往阻碍和模糊潜在的背景场景，导致视频监控和自动驾驶汽车等户外计算机视觉系统的准确性降低。本项目观察到两个物理特性：(i) 有雨图像中的背景场景不仅仅受雨水条纹的遮挡同时还受雨水带来的雾气遮挡。（ii）靠近相机的场景内物体主要受雨水条纹的影响而较远的物体受雾气的影响更多。现有的雨水条纹去除方法和数据集忽略了上述雨水相关的物理特性，从而降低了在真实图像中算法的准确性。本项目拟开发一个基于雨水物理特性和语义信息的单图雨水条纹去除框架，在消除雨水条纹同时更好地恢复雨水、雾气和噪声造成遮挡的背景场景细节。具体包括三个问题：1）基于雨水物理性质的雨水条纹消除。2）显著性区域背景场景细节增强。3）噪声大小自适应地图像降噪。本项目的研究为单图雨水条纹排除提供新的研究见解和新颖解决方案，并扩大其实际运

计算机视觉方法通常假设图像处理和场景理解的输入图像是清晰干净的。 然而雨水天气下采集的图像中的雨水条纹通常会降低图像质量且遮挡潜在的背景场景，从而降低视频监控和自动驾驶汽车等户外计算机视觉系统的准确性。在本项目中，我们观察到两个雨天的物理特性：(i) 有雨图像中的背景场景收到雨水条纹和雾气的同时遮挡。（ii）离相机近的场景内物体主要受雨水条纹的影响而较远的物体受雾气的影响更多。 但是现有的雨水条纹去除方法和数据集忽略了上述雨水相关的物理特性，从而降低了在真实图像中算法的准确性。为了解决这个问题，本项目研究基于雨水物理特性和语义信息的单图雨水条纹去除框架，合成对应的数据，在消除雨水条纹同时更好地恢复雨水、雾气和噪声遮挡的背景场景细节。具体包括三个问题：1）基于雨水物理性质的雨水条纹消除。2）显著性区域背景场景细节增强。3）噪声大小自适应地图像降噪。 因此，本项目围绕糟糕天气图像复原、噪声消除、以及显著性等相似级别的分割展开研究，同时研究的成果可以为单图雨水条纹消除提供新的研究见解和新颖解决方案，并扩大其实际运用。