融合灰度和深度特征的钢管表面缺陷检测方法

doi:10.3901/JME.2025.04.032

摘要/Abstract

摘要： 无缝钢管的表面缺陷检测与量化对于质量判定至关重要。然而，现有的检测方法主要依赖于灰度图像分析，缺乏对缺陷深度的综合判断，因此检测结果往往片面且不准确。为解决该问题，设计采用3D相机采集钢管表面的数据，能够同步获取到具备相同尺寸的灰度图像和点云数据，通过对点云数据的处理，可以计算出缺陷相对于基准表面的深度，并进一步量化得到伪彩色深度图像，能够直观地展示缺陷的深度信息。为了提升缺陷检测能力，在Yolov5模型的基础上添加双边网络结构，将灰度图像和伪彩色深度图像分别输入到细节分支和语义分支中提取特征，融合两分支的数据得到新的中间特征用于目标检测。试验结果表明，利用相对深度测量方法生成的伪彩色深度图像可以有效地消除抖动、扭转等情况的干扰，深度测量误差小于0.1 mm。此外，与传统的灰度图像检测模式相比，添加了双边网络结构的Yolov5模型在mAP指标上提升了4.7%，并且以108 帧/s的速率满足了实时检测的要求。最终在缺陷定性分析的基础上，通过增加深度方向维度，实现了对缺陷的定量分析，不仅提升了检测的全面性，也显著提高了检测的准确度。

关键词: 缺陷检测, 轮廓拟合, 伪彩色深度图像, Yolov5, 双边网络

Abstract: The detection and quantification of surface defects in seamless steel pipes are crucial for quality judgment. However, existing detection methods mainly rely on grayscale image analysis and lack comprehensive judgment of defect depth, resulting in one-sided and inaccurate detection results. To solve this problem, 3D cameras are used to collect data on the surface of steel pipes, which can synchronously obtain grayscale images and point cloud data with the same size. By processing the point cloud data, the depth of defects relative to the reference surface can be calculated, and further quantified to obtain pseudo color depth images, which can intuitively display the depth information of defects. In order to improve the defect detection capability, a bilateral network structure is added to the Yolov5 model, where grayscale images and pseudo color depth images are input into the detail branch and semantic branch respectively to extract features. The data from the two branches are fused to obtain new intermediate features for object detection. The experimental results show that the pseudo color depth image generated by the relative depth measurement method can effectively eliminate interference such as jitter and torsion, and the depth measurement error is less than 0.1 mm. In addition, compared with the traditional grayscale image detection mode, the Yolov5 model with the addition of a bilateral network structure improved the mAP index by 4.7% and met the real-time detection requirements at a speed of 108 fps. On the basis of qualitative analysis of defects, quantitative analysis of defects was achieved by adding depth dimension, which not only improved the comprehensiveness of detection but also significantly improved the accuracy of detection.

Key words: defect detection, contour fitting, pseudo-color depth images, Yolov5, bilateral network

中图分类号:

TP391

石杰, 吴昆鹏, 杨朝霖, 邓能辉, 王少聪, 苏成. 融合灰度和深度特征的钢管表面缺陷检测方法[J]. 机械工程学报, 2025, 61(4): 32-43.

SHI Jie, WU Kunpeng, YANG Chaolin, DENG Nenghui, WANG Shaocong, SU Cheng. Surface Defect Detection Method for Steel Pipes Based on Fusion of Gray and Depth Features[J]. Journal of Mechanical Engineering, 2025, 61(4): 32-43.

参考文献

[1] WANG Mingzhu，CHENG J C P. A unified convolutional neural network intergraded with conditional random field for pipe defect segmentation[J]. Computer-Aided Civil and Infrastructure Engineering，2020，35(2):162-177.
[2] 马涛，孙振国，陈强. 基于几何与纹理特征相融合的磁粉探伤裂纹提取算法[J]. 清华大学学报，2018，58(1):50-54. MA Tao，SUN Zhenguo，CHEN Qiang. Crack detection algorithm for fluorescent magnetic particle inspection based on shape and texture features[J]. Journal of Tsinghua University，2018，58(1):50-54.
[3] 周德强，潘萌，常祥，等. 铁磁性构件缺陷的脉冲涡流检测模式研究[J]. 仪器仪表学报，2017，38(6):1498-1505. ZHOU Deqiang，PAN Meng，CHANG Xiang，et al. Research on detection modes of ferromagnetic component defects using pulsed eddy curren[J]. Chinese Journal of Scientific Instrument，2017，38(6):1498-1505.
[4] 周正干，孙广开. 先进超声检测技术的研究应用进展[J]. 机械工程学报，2017，53(22):1-10. ZHOU Zhenggan，SUN Guangkai. New progress of the study and application of advanced ultrasonic testing technology[J]. Journal of Mechanical Engineering，2017，53(22):1-10.
[5] 李可，祁阳，宿磊，等. 基于改进ACGAN的钢表面缺陷视觉检测方法[J]. 机械工程学报，2022，58(24):32-40. LI Ke，QI Yang，SU Lei，et al. Visual inspection method for steel surface defects based on improved ACGAN[J]. Journal of Mechanical Engineering，2022，58(24):32-40.
[6] REN Shaoqing，HE Kaiming，GIRSHICK R，et al. Faster R-CNN:Towards real-time object detection with region proposal networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence，2017，39(6):1137-1149.
[7] 温银堂，付凯，张玉燕，等. 基于超分辨率改进Faster R-CNN的点阵结构内部缺陷判识方法[J]. 机械工程学报，2022，58(21):266-273. WEN Yintang，FU Kai，ZHANG Yuyan，et al. A method of improved faster R-CNN for detecting internal defect of metal lattice structure based on super-resolution reconstruction[J]. Journal of Mechanical Engineering，2022，58(21):266-273.
[8] HE Kaiming，GKIOXARI G，DOLLAR P，et al. Mask R-CNN[C]//Proceedings of the IEEE International Conference on Computer Vision (ICCV)，Venice，Italy，Piscataway，NJ:IEEE，2017:2961-2969.
[9] 李晖晖，周康鹏，韩太初. 基于CReLU和FPN改进的SSD舰船目标检测[J]. 仪器仪表学报，2020，41(4):183-190. LI Huihui，ZHOU Kangpeng，HAN Taichu. Improved SSD ship target detection based on CReLU and FPN[J]. Journal of Instrumentation，2020，41(4):183-190.
[10] REDMON J，DIVVALA S，GIRSHICK R，et al. You only look once:unified，real-time object detection[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)，Las Vegas，NV，USA，Piscataway，NJ:IEEE，2016:779-788.
[11] 程婧怡，段先华，朱伟. 改进YOLOv3的金属表面缺陷检测研究[J]. 计算机工程与应用，2021，57(19):252-258. CHENG Jingyi，DUAN Xianhua，ZHU Wei. Research on metal surface defect detection by improving YOOv3[J]. Computer Engineering and Application，2021，57(19):252-258.
[12] PAN Huihui，SUN Weichao，SUN Qiming，et al. Deep learning based data fusion for sensor fault diagnosis and tolerance in autonomous vehicles[J]. Chinese Journal of Mechanical Engineering，2021，34:72.
[13] 梁忠超，黄茁，胡兴，等. 基于YOLO网络的车辆四点标定测距研究[J]. 机械工程学报，2023，59(10):226-235. LIANG Zhongchao，HUANG Zhuo，HU Xing，et al. Research on distance measurement using vehicle four-point calibration based on YOLO neural network[J]. Journal of Mechanical Engineering，2023，59(10):226-235.
[14] 田壮壮，张恒伟，王坤，等. 改进CenterNet在遥感图像目标检测中的应用[J]. 遥感学报，2023，27(12):2706-2715. TIAN Zhuangzhuang，ZHANG Hengwei，WANG Kun，et al. Application of an improved CenterNet in remote sensing images object detection[J]. National Remote Sensing Bulletin，2023，27(12):2706-2715.
[15] TIAN Zhi，SHEN Chunhua，CHEN Hao，et al. FCOS:Fully convolutional one-stage object detection[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision. Seoul，Korea (South)，Piscataway:IEEE，2019:9627-9636.
[16] WANG Hao，LI Mengjiao，WAN Zhibo. Rail surface defect detection based on improved Mask R-CNN[J]. Computers and Electrical Engineering，2022，102:108269.
[17] FU Guizhong，SUN Peize，ZHU Wenbin，et al. A deep-learning-based approach for fast and robust steel surface defects classification[J]. Optics and Lasers in Engineering，2019，121:397-405.
[18] KOU Xupen，LIU Shuaijun，CHENG Kaiqiang，et al. Development of a YOLO-V3-based model for detecting defects on steel strip surface[J]. Measurement，2021，182:109454.
[19] LI Mengjiao，WANG Hao，WAN Zhibo. Surface defect detection of steel strips based on improved YOLOv4[J]. Computers and Electrical Engineering，2022，102:108208.
[20] YING Zhiping，LIN Zhongping，WU Zhenyu，et al. A modified-YOLOv5s model for detection of wire braided hose defects[J]. Measurement，2022，190:110683.
[21] CHARLES R Q，HAO Su，MO Kaichun，et al. PointNet:Deep learning on point sets for 3D classification and segmentation[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)，Honolulu，HI，USA，Piscataway，NJ:IEEE，2017:652-660.
[22] HU Xue，BAENA F R Y. Automatic bone surface restoration for markerless computer-assisted orthopaedic surgery[J]. Chinese Journal of Mechanical Engineering，2022，35:18.
[23] QI C R，LI Yi，HAO Su，et al. PointNet++:Deep hierarchical feature learning on point sets in a metric space[C]//Advances in Neural Information Processing Systems，Long Beach California USA，New York，NY:Curran Associates，2017:5105-5114.
[24] THOMAS H，QI C R，DESCHAUD J E，et al. KPConv:Flexible and deformable convolution for point clouds[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)，Seoul，Korea (South)，Piscataway，NJ:IEEE，2019:6411-6420.
[25] HU Qingyong，YANG Bo，XIE Linhai，et al. RandLA-Net:Efficient semantic segmentation of large-scale point clouds[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)，Seattle，WA，USA，Piscataway，NJ:IEEE，2020:11108-11117.
[26] YU Changqian，GAO Changxin，WANG Jingbo，et al. BiSeNet V2:Bilateral network with guided aggregation for real-time semantic segmentation[J]. International Journal of Computer Vision，2021，129:3051-3068.
[27] CHOLLET F. Xception:Deep learning with depthwise separable convolutions[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu，HI，USA，Piscataway，NJ:IEEE，2017:1800-1807.
[28] CHEN L C，PAPANDREOU G，KOKKINOS I，et al. DeepLab:Semantic image segmentation with deep convolutional nets，atrous convolution，and fully connected crFs[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence，2018，40(4):834-848.
[29] GAO Junbo，LIANG Junru，LI Junlong，et al. White-light endoscopic colorectal lesion detection based on improved YOLOv7[J]. Biomedical Signal Processing and Control，2024，90:105897.
[30] 郎宁，王德成，程鹏. 基于集成自适应欠采样的铝管表面缺陷检测方法研究[J]. 机械工程学报，2023，59(6):18-31. LANG Ning，WANG Decheng，CHENG Peng. Research on surface defect detection method of aluminum tube based on ensemble adaptive undersampling[J]. Journal of Mechanical Engineering，2023，59(6):18-31.