基于远红外与可见光图像深度融合特征的全天候行人检测方法研究

Unmanned vehicles can provide safe and efficient travel services for people, and also play an increasingly important role for social and economic development. Pedestrian detection is a key step in the safety driving of unmanned vehicles, and it is also a hot and difficult scientific problem in the computer vision field. In nowdays, all-day pedestrian detection is an important challenge problem for unmanned driving system. This proposal focuses on the on-vehcile all-day pedestrian detection system which can meet the application requirements. It makes use of the complementary information of far-infrared and visible image to carry out the following research items: (1) To establish feature fusion model of multimodal images with collaborative learning schema, focus on designing efficient detection algorithm from multi-modality differential feature; (2) To establish multi-granularity feature fusion model with collaborative learning schema, focusing on effective scale guided pedestrian detection algorithm; (3) To study pedestrian detection and segmentation problem based on weak supervised human segmentation information, and try to learn the interaction mechanism between pedestrian detection and segmentation tasks. (4) In addition, on the basis of the above, the object relationship model is established to find the mechenism of how scene semantic information providing evidence support for candidate pedestrians. The expected results of this proposal can not only enrich the theoretical research of multi-modality image fusion, but also promote the pedestrian detection technology and improve the safety performance of unmanned vehicles.

无人车能为人们提供安全高效的出行服务，也对社会经济发展具有越来越重要的引导作用。行人检测是无人车安全行驶的关键步骤，也是计算机视觉领域的研究热点和难点，而全天候行人检测是当前无人驾驶面临的重要挑战。本项目面向全天候车载行人检测任务的应用需求，利用远红外与可见光图像的互补信息，对以下四个方面展开研究：⑴建立基于多模态图像的特征融合、协同学习模型，研究基于多模态差分特征的高效检测算法；⑵建立基于多模态图像的多粒度特征融合、自适应区域特征提取协同学习模型，研究尺度信息引导的鲁棒行人检测算法；⑶在此基础上，建立基于弱监督人体分割信息的行人检测、分割协同学习模型，研究行人检测与分割任务之间的信息交互机制；⑷在以上基础上，建立目标关联关系模型，研究语义场景对行人检测任务的证据支撑机制。本项目的研究成果不仅可以丰富多模态图像融合的理论体系，而且将显著推动行人检测技术研究，提升无人车安全性能。

本项目以多光谱行人检测任务作为研究目标，基于数据驱动的多模态图像和基于知识驱动的先验信息相互融合、协同学习作为研究主线，将先验信息、场景语义信息提取和多模态图像深度特征融合方法作为研究切入点，着重研究以下四个方面：⑴ 基于弱分割信息引导的多光谱图像深度特征融合学习模型；⑵ 基于无锚框结构的局部-全局混合注意力机制的多模态图像深度特征融合学习模型；⑶ 基于自注意力机制和空间推理的多粒度特征聚合学习模型；⑷具有尺度感知能力的排列注意力机制和相邻特征聚合的特征融合模型。上述研究成果有望在物体检测，物体跟踪，行为识别等诸多实际应用领域。基于项目成果，在国内外学术期刊发表论文 7 篇（SCI 收录 6 篇、EI 收录 6 篇）；在国内外学术会议上发表论文 3 篇(EI 收 录 2 篇)：申请发明专利 4 项。