Surface Inspection of Polished Rail Welds Based on Photometric Stereo

doi:10.3901/JME.2024.24.011

Abstract

Abstract: A multi-surface information acquisition system for rails is designed to realize the image sequence capture of photometric stereo under multiple illumination directions. Normal estimation of grinding surfaces is implemented based on an excellent photometric stereo method. By analyzing the characteristics of polished surface normals and defect features, an adaptive polishing region extraction method is first proposed to eliminate the interference of the background areas. Secondly, a horizontal pattern defect detection algorithm based on two-stage and rotating rectangle extraction and a height mutation defect detection algorithm based on dual-threshold segmentation of grayscale histogram is proposed, which overcomes the noise and pseudo defect interference and effectively solves the problem of difficulty in defect feature extraction and localization. Experimental results demonstrate the superiority of the proposed polished surface defect detection algorithm over existing deep learning-based defect detection algorithms. The detection accuracy of the horizontal pattern reaches 94.4%, with a low omission rate of 0.84%. The accuracy of height mutation detection is 92.64%, with a low omission rate rate of 1.56%. The overall operation efficiency of the system is 23.784 s per rail welding surface, which meets the actual production requirements.

Key words: photometric stereo, defect detection, rail weld grinding surface, normal features, feature extraction

CLC Number:

TP391

WEI Xiaoyao, LIU Wenyuan, WANG Lingyu, DING Binjie, CAO Yanlong, YANG Jiangxin, HUANG Fang. Surface Inspection of Polished Rail Welds Based on Photometric Stereo[J]. Journal of Mechanical Engineering, 2024, 60(24): 11-24.

Add to citation manager EndNote|Reference Manager|ProCite|BibTeX|RefWorks

URL: http://www.cjmenet.com.cn/EN/10.3901/JME.2024.24.011

http://www.cjmenet.com.cn/EN/Y2024/V60/I24/11

References

[1] HUANG Yun，WEI Ming，XIAO Guijian，et al. Allowance extraction considering of inner and outer contour and experimental research on belt grinding of hollow blade[J]. Chinese Journal of Mechanical Engineering，2023，36(1)：1-11.
[2] 孙燕华，康宜华，石晓鹏. 基于单一轴向磁化的钢管高速漏磁检测方法[J]. 机械工程学报，2010，46(10)：8-13.SUN Yanhua，KANG Yihua，SHI Xiaopeng. High-speed MFL testing method for steel pipe under unidirectional axial magnetization[J]. Journal of Mechanical Engineering，2010，46(10)：8-13.
[3] 陈国龙，曹政，张帅帅，等. 基于科赫差测量平面涡流传感器的裂纹检测性能研究[J]. 机械工程学报，2023，59(10)：17-29.CHEN Guolong，CAO Zheng，ZHANG Shuaishuai，et al. Study on the crack detectability based on koch differential pick-up planar eddy current probe[J]. Journal of Mechanical Engineering，2023，59(10)：17-29.
[4] 李天骥，石永生，陈峰，等. 基于超声相控阵全聚焦DAC图谱的钢轨缺陷定量方法[J]. 机械工程学报，2021，57(18)：32-41.LI Tianji，SHI Yongsheng，CHEN Feng，et al. Quantitative method of rail flaws based on ultrasonic phased array and total focusing DAC mappings[J]. Journal of Mechanical Engineering，2021，57(18)：32-41.
[5] JHA S B，BABICEANU R F. Deep CNN-based visual defect detection：Survey of current literature[J]. Computers in Industry，2023，148：103911.
[6] 马骙骙，杨新文，张昭，等. 基于结构光视觉的钢轨波磨检测与定位方法研究[J]. 机械工程学报，2022，58(22)：140-147.MA Kuikui，YANG Xinwen，ZHANG Zhao，et al. Research on detection and positioning method of rail corrugation based on structural light vision[J]. Journal of Mechanical Engineering，2022，58(22)：140-147.
[7] 张仕军，金振林. 基于多模型级联的双目视觉铸件缺陷检测方法[J]. 机械工程学报，2022，58(5)：34-43.ZHANG Shijun，JIN Zhenlin. Casting defect detection method based on multi model cascade and binocular vision[J]. Journal of Mechanical Engineering，2022，58(5)：34-43.
[8] ZHENG Qian，SHI Boxin，PAN Gang. Summary study of data-driven photometric stereo methods[J]. Virtual Reality and Intelligent Hardware，2020，2(3)：213-221.
[9] ZHOU S K，AGGARWAL G，CHELLAPPA R，et al. Appearance characterization of linear lambertian objects，generalized photometric stereo，and illumination-invariant face recognition[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence，2007，29(2)：230-245.
[10] MIYAZAKI D，HARA K，IKEUCHI K. Median photometric stereo as applied to the segonko tumulus and museum objects[J]. International Journal of Computer Vision，2010，86：229-242.
[11] 蔡轶珩，张琳琳，盛楠，等. 基于光度立体法的中医舌体三维表面重建[J]. 电子与信息学报，2015，37(11)：2564-2570.CAI Yiheng，ZHANG Linlin，SHENG Nan，et al. 3D reconstruction of tongue surface based on photometric stereo method[J]. Journal of Electronics & Information Technology，2015，37(11)：2564-2570.
[12] 刘根. 基于光度立体视觉和图像显著性的皮革缺陷检测[D]. 广州：广东工业大学，2018.LIU Gen. Leather defect detection based on photometric stereo and image saliency[D]. Guangzhou：Guangdong University of Technology，2018.
[13] 朱可，霍彦文，武通海，等. 基于光度立体视觉三维重构算法的微观磨损形貌原位测量原理及方法[J]. 机械工程学报，2021，57(10)：1-9.ZHU Ke，HUO Yanwen，WU Tonghai，et al. Principle and method for in-situ measurement of micro-scale worn surface morphology based on 3D reconstruction with photometric stereo vision algorithm[J]. Journal of Mechanical Engineering，2021，57(10)：1-9.
[14] LEE J H，OH H M，KIM M Y. Deep learning based 3D defect detection system using photometric stereo illumination[C]//2019 International Conference on Artificial Intelligence in Information and Communication (ICAIIC)，February 11-13，2019，Okinawa，Japan. NewYork：IEEE，2019：484-487.
[15] KANG D，JANG Y J，WON S. Development of an inspection system for planar steel surface using multispectral photometric stereo[J]. Optical Engineering，2013，52(3)：039701.
[16] 韩阳. 航天密封圈视觉成像及缺陷检测技术研究[D]. 南京：南京理工大学，2018.HAN Yang. Research on visual imaging and defect detection technology for aerospace seals[D]. Nanjing：Nanjing University of Science and Technology，2018.
[17] 李正林. 应用于3D内窥镜的光度立体重建技术研究[D]. 杭州：浙江大学，2017.LI Zhenglin. The research of photometric stereo reconstruction applied to 3D endoscopy[D]. Hangzhou：Zhejiang University，2017.
[18] PATRICIO M A，MARAVALL D. A novel generalization of the gray-scale histogram and its application to the automated visual measurement and inspection of wooden Pallets[J]. Image and Vision Computing，2007，25(6)：805-816.
[19] CHAUDHARI C V. Steel surface defect detection using glcm，gabor wavelet，hog，and random forest classifier[J]. Turkish Journal of Computer and Mathematics Education (TURCOMAT)，2021，12(12)：263-273.
[20] 李奕璠，杨杰，陈再刚，等. 基于分层算子形态小波的轮对轴承复合故障检测[J]. 机械工程学报，2022，58(10)：1-11.LI Yifan，YANG Jie，CHEN Zaigang，et al. Wheelset-bearing compound fault detection based on layered-operator morphological wavelet[J]. Journal of Mechanical Engineering，2022，58(10)：1-11.
[21] LIU Xiaojing，XUE Feng，TENG Lu. Surface defect detection based on gradient LBP[C]//2018 IEEE 3rd International Conference on Image，Vision and Computing (ICIVC)，27-29 June 2018，Chongqing，China. NewYork：IEEE，2018：133-137.
[22] RAHEJA J L，KUMAR S，CHAUDHARY A. Fabric defect detection based on GLCM and Gabor filter：A comparison[J]. Optik，2013，124(23)：6469-6474.
[23] 汤勃，孔建益，伍世虔. 机器视觉表面缺陷检测综述[J]. 中国图象图形学报，2017，22(12)：1640-1663.TANG Bo，KONG Jianyi，WU Shiqian. Review of surface defect detection based on machine vision[J]. Journal of Image and Graphics，2017，22(12)：1640-1663.
[24] ZHAO Zhibin，WU Jingyao，LI Tianfu，et al. Challenges and opportunities of AI-enabled monitoring，diagnosis & prognosis：A review[J]. Chinese Journal of Mechanical Engineering，2021，34(1)：1-29.
[25] CAESARENDRA W，WIDODO A，YANG B S. Application of relevance vector machine and logistic regression for machine degradation assessment[J]. Mechanical Systems and Signal Processing，2010，24(4)：1161-1171.
[26] YILDIZ K，BULDU A，DEMETGUL M. A thermal-based defect classification method in textile fabrics with K-nearest neighbor algorithm[J]. Journal of Industrial Textiles，2016，45(5)：780-795.
[27] DAI Juying，TANG Jian，HUANG Shuzhan，et al. Signal-based intelligent hydraulic fault diagnosis methods：Review and prospects[J]. Chinese Journal of Mechanical Engineering，2019，32(1)：75.
[28] 周澄，邓菲，刘尧，等. 基于神经网络和支持向量机的导波弯管腐蚀损伤程度辨识研究[J]. 机械工程学报，2021，57(12)：136-144.ZHOU Cheng，DENG Fei，LIU Yao，et al. Identification of corrosion damage degree of guided wave bend pipe based on neural network and support vector machine[J]. Journal of Mechanical Engineering，2021，57(12)：136-144.
[29] BHATT P M，MALHAN R K，RAJENDRAN P，et al. Image-based surface defect detection using deep learning：A review[J]. Journal of Computing and Information Science in Engineering，2021，21(4)：1-23.
[30] SIMONYAN K，ZISSERMAN A. Very deep convolutional networks for large-scale image recognition[C]//3rd International Conference on Learning Representations (ICLR 2015)，May 7-9，2015，San Diego，CA，USA. CA：Computational and Biological Learning Society，2015：1-14.
[31] HE Kaiming，ZHANG Xiangyu，REN Shaoqing，et al. Deep residual learning for image recognition[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，June 27-30，2016，Las Vegas，NV，USA. Las Vegas：IEEE，2016：770-778.
[32] SANDLER M，HOWARD A，ZHU M，et al. Mobi- lenetv2：Inverted residuals and linear bottlenecks[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，June 18-23，2018，Salt Lake City，UT，USA. Salt Lake City：IEEE，2018：4510-4520.
[33] TAN Mingxing，LE Quoc. Efficientnet：Rethinking model scaling for convolutional neural networks[C]// International Conference on Machine Learning，PMLR，June 9-15，Long Beach，California，USA. Long Beach：ACM，2019：6105-6114.
[34] LIU Wei，ANGUELOV D，ERHAN D，et al. Ssd：Single shot multibox detector[C]//Computer Vision–ECCV 2016：14th European Conference，October 11-14，2016，Amsterdam，The Netherlands：Springer International Publishing，2016：21-37.
[35] JIANG Peiyuan，ERGU D，LIU Fangyao，et al. A Review of Yolo algorithm developments[J]. Procedia Computer Science，2022，199：1066-1073.
[36] LIN T Y，GOYAL P，GIRSHICK R，et al. Focal loss for dense object detection[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence，2020，42(2)：318-327.
[37] REN Shaoqing，HE Kaiming，GIRSHICK R，et al. Faster R-CNN：Towards real-time object detection with region proposal networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence，2017，39(6)：1137-1149.
[38] 罗东亮，蔡雨萱，杨子豪，等. 工业缺陷检测深度学习方法综述[J]. 中国科学：信息科学，2022，52(6)：1002-1039.LUO Dongliang，CAI Yuxuan，YANG Zihao，et al. Survey on industrial defect detection with deep learning[J]. SCIENTIA SINICA Informationis，2022，52(6)：1002-1039.
[39] CHEN Guangying，HAN Kai，SHI Boxin，et al. Deep photometric stereo for non-lambertian surfaces[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence，2020，44(1)：129-142.
[40] SANTO H，SAMEJIMA M，SUGANO Y，et al. Deep photometric stereo networks for determining surface normal and reflectances[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence，2020，44(1)：114-128.
[41] CAO Y，DING B，HE Z，et al. Learning inter-and intraframe representations for non-Lambertian photometric stereo[J]. Optics and Lasers in Engineering，2022，150：106838.
[42] SHI Boxin，MO Zhiping，WU Zhe，et al. A benchmark dataset and evaluation for non-lambertian and uncalibrated photometric stereo[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence，2019，41(2)：271-284.
[43] CAO Yanlong，WEI Xiaoyao，LIU Wenyuan，et al. A novel learning based non-lambertian photometric stereo method for pixel-level normal reconstruction of polished surfaces[J]. Machines，2022，10(2)：120.
[44] WOODHAM R J. Photometric method for determining surface orientation from multiple images[J]. Optical Engineering，1980，19(1)：139-144.
[45] OTSU N. A threshold selection method from gray-level histograms[J]. IEEE Transactions on Systems，Man，and Cybernetics，1979，9(1)：62-66.
[46] SHAIKH M S，CHOUDHRY A，WADHWANI R. Analysis of digital image filters in frequency domain[J]. International Journal of Computer Applications，2016，140(6)：12-19.
[47] KANOPOULOS N，VASANTHAVADA N，BAKER R L. Design of an image edge detection filter using the Sobel operator[J]. IEEE Journal of Solid-state Circuits，1988，23(2)：358-367.
[48] 程鹏飞，闫浩文，韩振辉. 一个求解多边形最小面积外接矩形的算法[J]. 工程图学学报，2008，29(1)：122-126.CHENG Pengfei，YAN Haowen，HAN Zhenhui. An algorithm for computing the minimum area bounding rectangle of an arbitrary polygon[J]. Journal of Engineering Graphics，2008，29(1)：122-126.