基于视觉注意机制和深度学习的多相机多模态多类型钢板表面微小缺陷检测与识别

The surface defects detection of steel plate based on machine vision is a hot topic at present. The steel plate surface defects have the characteristics of various kinds, various shapes, small size, as well as faint edge. The detection method of machine vision often has poor real-time performance, low efficiency, low rate of detection and classification, and so on. And the present detection method does not integrate the three-dimensional depth information of the steel plate defects with the two-dimensional information such as gray and texture. The multi-source and multi-camera alternately strobe synchronous camera method is constructed to obtain the two-dimensional and three-dimensional information of the image simultaneously. It is the idea of this research to introduce biological visual attention mechanism and build deep learning method, and detect surface defects based on multi-scale, multi-modal and multi vision features. With biological mechanism of visual attention, the advanced knowledge will be introduced to guide the visual attention according to the visual task. The integrate task relevant visual attention will be formed combining with bottom-up and top-down perception mechanism. The research of feature extraction, fusion and target detection for steel plate surface images will be carried out, and the limited information form massive data is selected effectively. Deep learning model based on multi-scale convolution recursive neural network can automatically learn complex high-level features in mass data, and reduce the complexity of manual feature design, and so get more abstract high-level features. Then, support vector machine is applied to realize fast and effective recognition and classification of multi-class micro defects on steel plate.

基于机器视觉的钢板表面缺陷检测是目前研究的一个热点。钢板表面缺陷种类繁多、形态多样、尺寸微小、边缘特征不明显，致使机器视觉的检测方法存在实时性差、效率不高、检测和识别率不佳等问题，且目前的检测方法没有将钢板表面缺陷的灰度、纹理等二维信息与三维深度信息融合。构建多光源、多相机交替频闪同步摄像方法同时获得图像的二维灰度信息与三维深度信息，引入生物视觉注意机制，构建深度学习方法，基于多尺度、多模态、多视觉特征的钢板表面缺陷检测是本项目的思路。根据视觉任务，结合自底向上的注意和自顶向下的注意形成完整的任务相关视觉注意机制，开展图像的视觉特征提取、融合和检测研究，进行海量数据中有限信息的筛选。采用多尺度卷积递归神经网络的深度学习模型，自动地学习海量数据中复杂的高层次特征、降低手工特征设计的复杂度，从而得到更抽象的高层特征，再由支持向量机实现钢板表面多类型微小缺陷的快速有效识别分类。

本项目从成像方法、图像增强、特征提取、目标分类和识别等方面对钢板表面缺陷的机器视觉检测进行了系统的研究，主要研究内容和取得的成果如下。.（1）构建了多相机、多光源、多模态的钢板表面缺陷检测实验平台，可以实现多种模式的图像采集，从而获取钢板表面缺陷图像的二维灰度信息与三维深度信息。在此基础上，研究了复合成像系统的标定。.（2）研究了钢板表面低对比度微小缺陷图像的增强方法和非均匀光照条件下的钢板表面缺陷图像的校正方法，以此突出了表面缺陷的细节信息，并有效解决了钢板表面缺陷图像光照不均匀问题。.（3）提出了自底向上和自顶向下相结合的注意机制算法，融合了钢板表面缺陷的底层和高层显著性特征（High-level saliency map），准确地分割出了多种缺陷类型的目标区域，获得了缺陷的位置信息，并采用Support Vector Machine (SVM)对缺陷目标区域进行分类。.（4） 提取了钢板表面缺陷图像的多尺度特征，并采用遗传算法进行降维，再采用SVM（Support Vector Machine）实现了小样本条件下的分类。.（5）研究了深度学习的（Convolution Neural Network，CNN）模型，采用了改进的CNN两阶段和一阶段的检测钢板表面缺陷算法，降低了手工特征设计的复杂度和主观性，提高了检测的精度。提出了基于Swin Transformer、FPN和RPN相结合的钢板表面缺陷检测模型，实现了高精度的多类型表面缺陷的端对端检测。.（6）研究了基于点云数据的钢板表面三维缺陷检测方法，提出了改进的点云精简、移除离群点和去噪算法，实现了钢板表面缺陷点云分割。.在国内外重要刊物发表/录用论文18篇，授权发明专利2项、实用新型专利1项和软件著作权1项，培养硕士研究生9名、博士研究生1名和青年教师1名。