基于多模型级联的双目视觉铸件缺陷检测方法

doi:10.3901/JME.2022.05.34

机械工程学报 ›› 2022, Vol. 58 ›› Issue (5): 34-43.doi: 10.3901/JME.2022.05.34

扫码分享

基于多模型级联的双目视觉铸件缺陷检测方法

张仕军¹, 金振林^1,2

1. 燕山大学机械工程学院秦皇岛 066004;
2. 燕山大学河北省重型智能制造装备技术创新中心秦皇岛 066004

收稿日期:2021-03-01 修回日期:2021-07-23 出版日期:2022-03-05 发布日期:2022-04-28
通讯作者: 金振林(通信作者),男,1962年出生,博士,教授,博士研究生导师。主要研究方向为并联机器人技术及应用。E-mail:zljin@ysu.edu.cn E-mail:zljin@ysu.edu.cn
作者简介:张仕军,男,1993年出生,博士研究生。主要研究方向为计算机视觉与深度学习。E-mail:zhangshijun_ys@163.com
基金资助:
国家自然科学基金重大科研仪器研制(51927809);河北省自然科学基金面上(E2021203018);河北省2022年研究生创新(CXZZBS2022145)资助项目。

Casting Defect Detection Method Based on Multi Model Cascade and Binocular Vision

ZHANG Shi-jun¹, JIN Zhen-lin^1,2

1. Institute of Mechanical Engineering, Yanshan University, Qinhuangdao 066004;
2. Heavy-duty Intelligent Manufacturing Equipment Innovation Center of Hebei Province, Yanshan University, Qinhuangdao 066004

Received:2021-03-01 Revised:2021-07-23 Online:2022-03-05 Published:2022-04-28

摘要/Abstract

摘要： 为了解决铸件表面缺陷检测以及缺陷位置的三维定位问题,采用双目视觉系统获得图像,依靠三角测量法获得点云数据,基于正态分布变换配准方法实现工件的三维定位。通过多模型级联解决少样本和样本比例失衡情况下的表面缺陷检测,结合无监督模型的编码解码方式,依靠正样本训练获得对缺陷图像进行修复的网络模型,将修复后的图像与原始图像进行差异分析获得缺陷位置;使用MASK-RCNN模型进行监督模型训练,获得一体化检测分割模型,直接定位缺陷位置及类别。此外还将运动机构的物理坐标系、双目成像的工件坐标系以及平面图像的坐标进行换算,得到多个坐标系的转换关系,实现了平面缺陷的位置信息映射到三维工件的空间中。实验表明提出的双目视觉系统在工件成像以及检测方面具有良好的效果。

关键词: 双目视觉, 深度学习, 缺陷检测, 多模型级联

Abstract: In order to realize the surface defect detection and three-dimensional location of casting defects, we obtain the image by binocular vision system, obtain the point cloud data by triangulation method, and normal distribution transformation registration method is used to realize the three-dimensional positioning of the workpiece. Multi model cascading is used to solve the surface defect detection under the condition of few samples and unbalanced sample proportion. Combined with the coding and decoding method of unsupervised model, the network model of defect image repair is obtained by positive sample training, and the defect location is obtained by difference analysis between the repaired image and the original image; the supervised model training is carried out by MASK-RCNN model, and the integration is obtained detect the segmentation model and locate the defect location and category directly. In addition, the physical coordinate system of the moving mechanism, the workpiece coordinate system of binocular imaging and the coordinate of the plane image are converted to obtain the conversion relationship of multiple coordinate systems, and the position information of the plane defect is mapped to the space of the three-dimensional workpiece. Experiments show that this system has good effect in workpiece imaging and detection.

Key words: binocular vision, deep learning, defect detection, multi model cascade

中图分类号:

TP751

张仕军, 金振林. 基于多模型级联的双目视觉铸件缺陷检测方法[J]. 机械工程学报, 2022, 58(5): 34-43.

ZHANG Shi-jun, JIN Zhen-lin. Casting Defect Detection Method Based on Multi Model Cascade and Binocular Vision[J]. Journal of Mechanical Engineering, 2022, 58(5): 34-43.

参考文献

[1] 康敬乐,丁苏沛,孙剑飞,等.中国低压铸造装备技术的发展与展望[J].中国铸造装备与技术,2016(4):1-11.KANG Jingle,DING Supei,SUN Jianfei,et al.Development and prospect of low-pressure casting equipment technology in china[J].China Foundry Machinery & Technology,2016(4):1-11.
[2] 李彬,汪诚,吴静,等.改进YOLOv4算法的航空发动机部件表面缺陷检测[J].激光与光电子学进展,2021,58(14):1415004.LI Bin,WANG Cheng,WU Jing,et al.Surface defect detection of aeroengine components based on improved YOLOv4 algorithm[J].Laser & Optoelectronics Progress,2021,58(14):1415004.
[3] 于重重,萨良兵,马先钦,等.基于度量学习的小样本零器件表面缺陷检测[J].仪器仪表学报,2020,41(7):214-223.YU Chongchong,SA Liangbing,MA Xianqin,et al.Few-shot parts surface defect detection based on the metric learning[J].Chinese Journal of Scientific Instrument,2020,41(7):214-223.
[4] 陈麒麟,王东兴,林建钢,等.冲压磨削平板类零件的表面缺陷检测[J].锻压技术,2020,45(6):168-174.CHEN Qilin,WANG Dongxing,LIN Jiangang,et al.Surface defect detection on stamping and grinding flat parts[J].Forging & Stamping Technology,2020,45(6):168-174.
[5] DI He,XU Ke,ZHOU Peng.Defect detection of hot rolled steels with a new object detection framework called classification priority network[J].Computers & Industrial Engineering,2019,128:290-297.
[6] 訾斌,尹泽强,李永昌,等.基于YOLO模型的柔索并联机器人移动构件快速定位方法[J].机械工程学报,2019,55(3):64-72.ZI Bin,YIN Zeqiang,LI Yongchang,et al.Fast mobile component location method for cable-driven parallel robots based on YOLO model[J].Journal of Mechanical Engineering,2019,55(3):64-72.
[7] ZHANG Dejin,ZOU Qin,LIN Hong,et al.Automatic pavement defect detection using 3D laser profiling technology[J].Automation in Construction,2018,96:350-365.
[8] JIAN Chuanxia,GAO Jian,AO Yinhui,et al.Automatic surface defect detection for mobile phone screen glass based on machine vision[J].Applied Soft Computing,2017,52:348-358.
[9] XIONG Zhimin,LI Qingquan,MAO Qingzhou,et al.A3D laser profiling system for rail surface defect detection[J].Sensors,2017,17(8):1791.
[10] 刘巍,李肖,李辉,等.基于双目视觉的数控机床动态轮廓误差三维测量方法[J].机械工程学报,2019,55(10):1-9.LIU Wei,LI Xiao,LI Hui,et al.Binocular vision-based three-dimensional contouring error calibration method for NC machine tools[J].Journal of Mechanical Engineering,2019,55(10):1-9.
[11] OLIVER H,LI Xiaodong.In situ real time defect detection of 3D printed parts[J].Additive Manufacturing,2017,17:135-142.
[12] 颜佳桂,李宏胜,任飞.基于SSD和改进双目测距模型的车辆测距方法研究[J].激光杂志,2020,41(11):42-47.YAN Jiagui,LI Hongsheng,REN Fei.Research on vehicle ranging method based on SSD algorithm and improved binocular ranging model[J].Laser Journal,2020,41(11):42-47.
[13] MAGNUSSON M.The three-dimensional normaldistributions transform:An efficient representation for registration,surface analysis,and loop detection[D].Örebro:Örebro University,2009.
[14] ZHAO Z,LI B,DONG R,et al.A surface defect detection method based on positive samples[C]//Pacific Rim International Conference on Artificial Intelligence.Springer,Cham,2018:473-481.
[15] HE K,GKIOXARI G,DOLLAR P,et al.Mask R-CNN[J].IEEE Trans.Pattern Anal.Mach.Intell.,2020,42(2):386-397.

基于多模型级联的双目视觉铸件缺陷检测方法

Casting Defect Detection Method Based on Multi Model Cascade and Binocular Vision

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

本文评价

[1]	邓堡元, 何赟泽, 王洪金, 张宏, 杨渊, 马敏敏, 牟欣颖, 杨瑞珍. 光伏电池图像序列的深度学习检测方法[J]. 机械工程学报, 2021, 57(8): 98-106.
[2]	陈锐锐, 王健, 蔡峡, 陶勇, 宫赫. 基于双目视觉技术的特高压输电线路导地线弧垂检测[J]. 电气工程学报, 2021, 16(4): 183-188.
[3]	谢琦, 王娜, 杨雅静, 段斌, 马德志, 有移亮, 吴素君. 含缺陷钢结构焊接接头检测与评估[J]. 机械工程学报, 2021, 57(24): 223-232.
[4]	宫文峰, 王元哲, 陈辉, WANG Danwei. 基于深度学习的无人驾驶汽车导航传感器异常诊断方法[J]. 机械工程学报, 2021, 57(24): 268-278.
[5]	赵子婧, 刘宏哲, 曹东璞. 基于Libra R-CNN改进的交通标志检测算法[J]. 机械工程学报, 2021, 57(22): 255-265.
[6]	王久健, 杨绍普, 刘永强, 文桂林. 一种基于空间卷积长短时记忆神经网络的轴承剩余寿命预测方法[J]. 机械工程学报, 2021, 57(21): 88-95.
[7]	金鹏, 黄浩, 刘检华, 刘少丽, 方玥, 何森, 戚慧志, 刘威. 多传感器信息融合的铁路扣件缺陷检测方法[J]. 机械工程学报, 2021, 57(20): 38-46.
[8]	廖林, 袁懋诞, 纪轩荣, 戴安帮. 钢轨中高频超声导波单模态激励技术研究[J]. 机械工程学报, 2021, 57(18): 23-31.
[9]	赵树煊, 张洁, 汪俊亮, 徐楚桥. 基于两阶段深度迁移学习的面料疵点检测算法[J]. 机械工程学报, 2021, 57(17): 86-97.
[10]	林添良, 姚瑜, 许文杰, 付胜杰, 任好玲, 陈其怀. 基于环境识别的电动工程机械无人驾驶行走方法[J]. 机械工程学报, 2021, 57(10): 42-49.
[11]	姜洪权, 贺帅, 高建民, 王荣喜, 高智勇, 王晓桥, 夏锋社, 程雷. 一种改进卷积神经网络模型的焊缝缺陷识别方法[J]. 机械工程学报, 2020, 56(8): 235-242.
[12]	周兴康, 余建波. 基于深度一维残差卷积自编码网络的齿轮箱故障诊断[J]. 机械工程学报, 2020, 56(7): 96-108.
[13]	李兴, 侯伟, 杨宇, 王鹏, 苏海军, 付铭明. 论电力基建业务中边缘计算的应用与实践[J]. 电气工程学报, 2020, 15(4): 153-160.
[14]	乔严程, 解社娟, 仝宗飞, 唐敬达, 许盼盼, 李骥, 李丽娟, 陈振茂. 基于交变磁场红外热像的磁性水凝胶缺陷检测[J]. 机械工程学报, 2020, 56(18): 1-6.
[15]	张勇, 郑贤德, 季明江, 林鑫, 邱静, 刘冠军. 基于鱼侧线感知原理和深度学习的水下平动目标方向识别[J]. 机械工程学报, 2020, 56(12): 231-239.