基于红外与可见光图像的双波段双视场立体视觉关键问题研究

Binocular stereo vision is a research hot spot in the field of computer vision, and its research results are widely used in the field of industrial detection, robot navigation and so on. However, the performance of the single band sensor is limited by the wavelength response range, which can not meet the needs of the stereo vision application in more special environments. The imaging principles of infrared and visible images are different, and the advantages are complementary. The combination of the two methods in stereo vision is an effective way to solve the above problems. The research will be carried out on two aspects: First, improve the 3D reconstruction accuracy of stereo vision by using the redundancy of dual-band images information; second, expand the application field of dual-field stereo vision theory by using the complementary information of dual-band images. The preliminary work has set up hardware platform of the system (dual-field structure composed of four cameras), so the topics of the research mainly include: the precise registration scheme with dual-band images in the same field of view, the establishment of the aberration correction model in special circumstances, so as to ensure the accuracy of the system's calibration parameters; adaptive image fusion algorithm which is robust to environment changes, design an image fusion weight model based on machine learning; dual-band images stereo matching algorithm, a similarity measure function contains dual-band information and improved matching window, so as to reduce the false matching rate and improve the accuracy of the three-dimensional reconstruction. The research results will have good application prospects in the field of fire rescue, traffic monitoring, battlefield navigation and so on.

双目立体视觉是计算机视觉领域的研究热点，其研究成果在工业检测、机器人导航等领域应用广泛。然而单一波段传感器受波长响应的范围限制，无法满足更广泛环境中的立体视觉应用需求。红外与可见光图像的成像原理不同且优势互补，将二者结合应用于立体视觉，是解决以上问题的有效途径。课题拟从两个方面展开工作：一是利用双波段图像的信息冗余性提高立体视觉三维形貌重建精度；二是利用双波段图像的信息互补性扩展双视场立体视觉理论的应用领域。先期工作已初步搭建四目双视场的系统硬件平台，课题主要研究：同视场内双波段图像的精确配准方案，建立变参环境下的像差修正模型；具有环境变化鲁棒性的自适应图像融合算法，设计基于机器学习的图像融合权重模型；双波段双视场图像立体匹配算法，建立包含双波段信息的相似性测度函数和自适应匹配窗口，降低误匹配率，提高三维重建精度。课题研究成果在火灾救援、交通监测、战地导航等领域具有良好应用前景。

项目围绕红外与可见光双波段双目立体视觉关键问题进行了研究，全面分析和总结了基于异源图像互补性进行场景三维形貌重建所涉及的基本理论，同时研究了利用双波段图像信息冗余性提高三维重建精度的方法。.项目针对高温、烟雾、变光照环境下的双视场立体视觉技术的实现方法进行了探索。搭建了红外与可见光共光路的双视场立体视觉系统硬件模型，并优化了系统参数。针对红外光与可见光波长差异引起的变温环境下的双波段成像像差，设计了像差修正模型，保证同视场内红外与可见光图像精确配准，为三维点云的精确重建奠定了基础。针对大光照变化环境下的双波段图像融合问题，提出了具有环境变化鲁棒性的自适应融合算法，设计了多分辨率分析方法，对不同频率成分系数制定不同融合规则，有效保证不同光照环境下的图像融合质量。对于场景内多物体共存，因被测对象成像景深差异而导致的图像无法全聚焦问题，设计了基于相位拉伸变换的多聚焦图像融合算法，解决了图像失焦模糊的问题。利用双波段图像信息的冗余性，突破单一波段图像进行立体匹配时的信息量限制，建立了包含双波段信息的相似性测度函数及根据图像特点自适应变换的动态立体匹配窗口，有效提高了双视场图像立体匹配精度。项目后期尝试拓展课题研究内容，将声音信号与图像信号相结合，采用红外传感器、可见光传感器、麦克风传感器阵列，分别采集红外视觉信息，可见光视觉信息和声压信息，通过探索将声场三维点云与场景三维形貌点云的多源信息融合方法，为拓展项目应用前景奠定了基础，预期研究成果可应用于战地搜救，火灾救援等领域。