基于同地点场景稠密匹配的恶劣天气道路场景解析

Vehicles equipped with autonomous driving systems come into our life gradually. Road scene parsing plays an important role for them, i.e., assigning the semantic labels to every pixel in the road scene image. Road scene parsing methods perform well under good weather conditions. However, the performance is decreased sharply under drastic weather changes, e.g., rainy day, snowy day, foggy day, night, etc. Road scene parsing efficiently under these challenging scenarios is badly needed.. This project intends to proceed from the establishment of dense correspondences between road scene images of good weather and bad weather at the same place, and provide new ideas for solving road scene parsing problem under bad weather conditions. Firstly, the data set of same place images under various weather conditions for training deep dense matching network is built. Secondly, pixel regions of the same place road scene images under bad and good weather conditions(sunny days, etc.) are represented with robust deep features. Finally, the fast and effective dense correspondences of the pixel regions are established. To complete the road scene parsing under bad weather conditions, the semantic labels of road scene images under good weather conditions can be transferred to the images under bad weather conditions on the basis of the established dense correspondences.

完全自主驾驶车辆逐渐进入我们生活。其中，基于视觉道路场景解析是无人驾驶车辆重要组成部分，即给予道路场景图像中每个像素相应的语义标签。良好天气条件下，现有场景解析方法能较好地完成道路场景解析。然而，当途经恶劣天气（雨天、雪天、雾天、晚上等），现有方法性能则会大大下降。如何在恶劣天气条件下进行有效道路场景解析是亟待解决问题。. 本项目拟从建立同地点良好天气和恶劣天气道路场景图像之间稠密对应关系出发，为解决恶劣天气下道路场景解析问题提供新思路。首先，构建不同天气、同地点场景深度稠密匹配网络训练数据集。其次，对同地点恶劣天气和良好天气（晴天等）道路图像中像素区域进行鲁棒深度特征表示。最后，建立快速、有效的像素区域稠密对应关系。在稠密对应关系基础上则可把良好天气道路场景语义标签信息转移到恶劣天气道路图像，从而完成恶劣天气下道路场景解析。

良好天气条件下，现有场景解析方法能较好地完成场景解析。然而，当途经恶劣天气（雨天、雪天、雾天、晚上等），现有方法性能则会大大下降。如何在恶劣天气条件下进行有效场景解析是亟待解决问题。一方面，考虑到道路图像显著空间结构信息，提出同地点道路场景在语义分割结果层进行域适应的方法，从而更好地建立快速、有效的像素区域稠密对应关系。另一方面，考虑到同地点图像中像素区域需进行鲁棒深度特征表示。首先，设计了一个端到端自监督卷积子空间聚类网络，同时进行特征学习和子空间聚类。其次，提出一个图表示网络，基于RNN(Recurrent Neural Network)提取分割层特征，然后利用GCN(Graph Convolutional Network)进行特征聚合。最后，鉴于有效利用无标签数据和减弱错误标签带来负面影响，提出基于离散反馈的半监督学习方法。同时，通过结合表面和结构特征，提出一种针对抽象目标的视觉特征表示方式。