基于深度视觉知识表达的输电线路螺栓表面状态检测方法研究

Bolts are extremely important and numerous fasteners on transmission line and their good state ensure the safe operation of the power grid. This project uses the actual aerial images of transmission line as data source to study the automatic detection method of bolts surface state. Aiming at the problems such as the bolts’ small area ratio in the aerial image of transmission line, the lack of deep feature representation, the complex background and the serious interference in aerial images, the unbalanced sample distribution and so on, this project proposes a hierarchical detection framework that first detects fittings and then the surface state of the bolts. We construct public typical fittings detection database and bolts surface state detection database and propose a detection method of typical fittings based on multilayer feature fusion expression. By integrating the fittings-bolts visual relationship knowledge into the deep learning model, a deep visual knowledge representation method based on the CapsNet model and attention mechanism is proposed. Multistage refined feature expression method, adaptive transversal classifier and multiple strategy processing methods are proposed, and an automatic detection method of bolts surface state based on an improved Faster R-CNN model is achieved. The performance of the proposed methods are evaluated and compared by experimental and theoretical analysis. The research results and pre-research results obtained by the project team indicate the feasibility and effectiveness of the project. The project is suitable for the development requirements of computer vision, image processing and industrial optical inspection, and has important research value and broad application prospects.

螺栓是输电线路上极其重要且大量存在的紧固件，其良好状态保障着电网的安全运行。本项目以输电线路航拍图像为数据源，研究螺栓表面状态自动检测方法。针对螺栓在航拍图像中占比小、深度特征表达区分度不高、图像背景复杂且干扰严重和样本分布不平衡等问题，提出先检测金具目标、再检测螺栓表面状态的分级检测框架：构建公开的典型金具检测数据库和螺栓表面状态检测数据库；提出基于多层特征融合表达的典型金具检测方法；把金具-螺栓视觉关系知识融入深度模型，提出基于CapsNet模型的深度视觉知识表达方法；提出多级精细化特征表达方法、自适应横向分类器和多种策略处理方法等，实现基于改进Faster R-CNN模型的螺栓表面状态自动检测方法。通过实验和理论分析的手段评价和比较所提出方法的性能。本项目组已取得的研究成果和预研结果表明了本项目的可行性和有效性，适应了计算机视觉、工业光学检测的发展要求，具有重要的研究价值和应用前景。

螺栓是输电线路上极其重要且大量存在的紧固件，其良好状态保障着电网的安全运行。本项目以输电线路航拍图像为数据源，研究螺栓表面状态自动检测方法。依据研究目标，本项目完成了6个方面的研究工作。(a)构建了2个专业图像数据库。(b)设计并实现了基于GAN模型的小样本数据扩增方法。(c)提出一种结合KL散度和形状约束的Faster R-CNN典型金具检测方法；提出了一种嵌入遮挡关系模块的SSD输电线路典型金具检测方法；提出一种基于自动架构搜索FPN的金具及其缺陷检测方法；通过构建关系推理模块提出了一种基于关系推理的输电线路金具及其缺陷检测方法；利用共现连接矩阵和空间位置矩阵作为金具结构信息的表达方式，提出一种基于知识推理的输电线路金具检测方法。(d)提出了一种基于二级检测框架的典型金具中螺栓目标检测方法；设计了由粗粒度到细粒度的级联检测框架，从而提出了一种基于复杂知识表示的金具上螺栓缺陷级联检测方法。(e)为充分利用螺栓视觉表征和语义表征，提出了一种联合视觉-语义知识的螺栓表面状态多标签识别方法；为解决现有模型不能学习到螺栓细粒度局部特征的问题，提出了一种联合视觉-位置知识的螺栓表面状态多标签识别方法。(f)利用螺栓的视觉形态知识，提出一种基于知识表达的螺栓目标及其缺陷检测方法；提出一种嵌入双注意机制的Faster R-CNN航拍螺栓缺陷检测方法；改进PCL模型，优化自适应加权损失函数，提出了一种基于PCL弱监督模型的输电线路螺栓缺陷检测方法。项目组完成了预期的研究内容，达到了预期的研究目标，并取得了丰富的研究成果；本项目的顺利实施适应了输电线路巡检、计算机视觉、人工智能的发展要求。