基于目标检测的选区激光熔融成形过程熔池与飞溅监测

doi:10.3901/JME.2023.09.335

机械工程学报 ›› 2023, Vol. 59 ›› Issue (9): 335-348.doi: 10.3901/JME.2023.09.335

扫码分享

基于目标检测的选区激光熔融成形过程熔池与飞溅监测

毛杨坤^1,2, 段现银³, 林昕³, 傅盈西⁴, 朱锟鹏^1,5

1. 中国科学院合肥物质科学研究院智能机械研究所合肥 230031;
2. 中国科学技术大学合肥 230026;
3. 武汉科技大学冶金装备及其控制教育部重点实验室武汉 430081;
4. 新加坡国立大学苏州研究院苏州 215123;
5. 常州先进制造技术研究所常州 213164

收稿日期:2022-06-21 修回日期:2023-01-22 出版日期:2023-05-05 发布日期:2023-07-19
通讯作者: 朱锟鹏(通信作者),男,1977年出生,博士,研究员,博士研究生导师。主要从事精密制造与自动化相关领域的理论和应用研究。E-mail:zhukp@iamt.ac.cn E-mail:zhukp@iamt.ac.cn
作者简介:毛杨坤,男,1997年出生。主要从事增材制造及其自动化相关领域的理论和应用研究。E-mail:1323341343@qq.com
基金资助:
国家自然科学基金资助项目(51875379，52175481)。

Melt Pool and Spatter Monitoring in Selective Laser Melting Forming Process Based on Target Detection

MAO Yangkun^1,2, DUAN Xianyin³, LIN Xin³, FUH Y H J⁴, ZHU KunPeng^1,5

1. Institute of Intelligent Machines, Hefei Institutes of Physical Science, Chinese Academy of Sciences, Hefei 230031;
2. University of Science and Technology of China, Hefei 230026;
3. Key Laboratory of Metallurgical Equipment and Control Technology, Ministry of Education, Wuhan University of Science and Technology, Wuhan 430081;
4. National University of Singapore(Suzhou) Research Institute, Suzhou 215123;
5. Changzhou Institute of Advanced Manufacturing Technology, Changzhou 213164

Received:2022-06-21 Revised:2023-01-22 Online:2023-05-05 Published:2023-07-19

摘要/Abstract

摘要： 在选区激光熔融成形过程中,飞溅与熔池包含了能够体现加工质量的重要特征信息,从成形过程采集到的熔池图像中,获得这些信息,实现选区激光熔融的过程监测是近年来研究的重点之一。为了更加精确且有效地从图像中提取熔池和飞溅的信息,提出了一种基于YOLOv5目标检测模型,实现了对成形过程图像中飞溅与熔池的实时定位与捕获。首先,以YOLOv5s目标检测网络为基础,调整骨干网络的深度与宽度,修改检测头的数量。之后,引入自校正卷积与CBAM注意力机制模块,设计了新的特征整合结构,通过上述步骤,提升了网络的检测性能。将工业相机采集到的图像制作为目标检测数据集,进行模型的训练与测试,结果表明该网络能够从原始图像中对飞溅与熔池目标进行准确的定位,在具有良好的检测精度的同时,网络模型的参数量极少,更加符合工业应用的需求。网络的检测精度mAP@0.5:0.95达到了0.466,为基于图像的选区激光熔融过程监测提供了一种新的方法。

关键词: 选区激光熔融, 目标检测, 过程监测, 熔池图像

Abstract: In the process of selective laser melting, spatter and melt pool contain important information which can reflect the processing quality. It is one of the research emphases in recent years to obtain this information from the melt pool images, which are collected in the processing process, and then realize the process monitoring of selective laser melting. In order to extract the information of melt pool and spatter more accurately and effectively, a target detection model based on YOLOv5 is proposed to realize the real-time location and capture of the spatter and melt pool from the processing image. Firstly, based on YOLOv5s target detection network, the depth and width of backbone network are adjusted, and the number of detection heads is modified. After that, Self-calibrated convolutions and CBAM attention module are introduced to design a new feature integration structure. Through the above steps, the detection performance of the network is improved. The images collected by industrial camera are made into target detection datasets for model's training and testing. The results show that the network can accurately locate the spatter and melt pool targets from the original image. With better detection accuracy, the network model has few parameters, which is more in line with the needs of industrial applications. The detection accuracy of mAP@0.5:0.95 reaches 0.466, thus provides a new method for the monitoring of selective laser melting process based on images.

Key words: selective laser melting, object detection, process monitoring, melt pool image

中图分类号:

TG156

毛杨坤, 段现银, 林昕, 傅盈西, 朱锟鹏. 基于目标检测的选区激光熔融成形过程熔池与飞溅监测[J]. 机械工程学报, 2023, 59(9): 335-348.

MAO Yangkun, DUAN Xianyin, LIN Xin, FUH Y H J, ZHU KunPeng. Melt Pool and Spatter Monitoring in Selective Laser Melting Forming Process Based on Target Detection[J]. Journal of Mechanical Engineering, 2023, 59(9): 335-348.

参考文献

[1] ZHU K,FUH J Y H,LIN X. Metal-based additive manufacturing condition monitoring:A Review on machine learning based approaches[J]. IEEE/ASME Transactions on Mechatronics,2021:1-16.
[2] 张家莲,李发亮,张海军. 选区激光熔化技术制备金属材料研究进展[J]. 激光与光电子学进展,2019,56(10):35-44. ZHANG Jialian,LI Faliang,ZHANG Haijun. Research progress on preparation of metallic materials by selective laser melting[J]. Laser & Optoelectronics Progress,2019,56(10):35-44.
[3] Fé-PERDOMO I L,RAMOS-GREZ J A,BERUVIDES G,et al. Selective laser melting:Lessons from medical devices industry and other applications[J]. Rapid Prototyping Journal,2021,27(10):1801-1830.
[4] 杨永强,宋长辉,王迪. 激光选区熔化技术及其在个性化医学中的应用[J]. 机械工程学报,2014,50(21):140-151. YANG Yongqiang,SONG Changhui,WANG Di. Selective laser melting and its applications on personalized medical parts[J]. Journal of Mechanical Engineering,2014,50(21):140-151.
[5] ZHANG J,SONG B,WEI Q,et al. A review of selective laser melting of aluminum alloys:Processing,microstructure,property and developing trends[J]. Journal of Materials Science & Technology,2019,35(2):270-84.
[6] 杨益,党明珠,李伟,等. 激光选区熔化钛铝合金裂纹形成机理及抑制研究[J]. 机械工程学报,2020,56(3):181-188. YANG Yi,DANG Mingzhu,LI Wei,et al. Study on cracking mechanism and inhibiting process of TiAl alloys fabricated by selective laser melting[J]. Journal of Mechanical Engineering,2020,56(3):181-188.
[7] LIN X,ZHU K,FUH J Y H,et al. Metal-based additive manufacturing condition monitoring methods:From measurement to control[J]. ISA Transactions,2021,120:147-166.
[8] 曲睿智,黄良沛,肖冬明. 选择性激光熔化过程中熔池演变与金属飞溅特性数值模拟[J]. 航空学报,2022,43(4):405-424. QU Ruizhi,HUANG Liangpei,XIAO Dongming. Numerical simulation of melt pool evolution and metal spattering characterization during selective laser melting processing[J]. Acta Aeronautica et Astronautica Sinica,2022,43(4):405-424.
[9] 朱锟鹏,王齐胜,林昕,等. 基于熔池运动特征的选区激光熔融过程状态检测方法[J]. 计算机集成制造系统,2021,27(12):3403-3415. ZHU Kunpeng,WANG Qisheng,LIN Xin,et al. Selective laser melting process state monitoring method based on motion feature of melt pool[J]. Computer Integrated Manufacturing Systems,2021,27(12):3403-3415.
[10] 唐梓珏,刘伟嵬,颜昭睿,等. 基于熔池动态特征的金属激光熔化沉积形状精度演化行为研究[J]. 机械工程学报,2019,55(15):39-47. TANG Zijue,LIU Weiwei,YAN Zhaorui,et al. Study on evolution behavior of geometrical accuracy based on dynamic characteristics of molten pool in laser-based direct energy deposition[J]. Journal of Mechanical Engineering,2019,55(15):39-47.
[11] LI R,LIU J,SHI Y,et al. Balling behavior of stainless steel and nickel powder during selective laser melting process[J]. The International Journal of Advanced Manufacturing Technology,2012,59(9):1025-1035.
[12] SAMES W J,LIST F A,PANNALA S, et al. The metallurgy and processing science of metal additive manufacturing[J]. International Materials Reviews,2016,61(5):315-360.
[13] CUNNINGHAM R,ZHAO C,PARAB N,et al. Keyhole threshold and morphology in laser melting revealed by ultrahigh-speed x-ray imaging[J]. Science,2019,363(6429):849-852.
[14] TAHERI ANDANI M,DEHGHANI R,KARAMOOZ-RAVARI M R,et al. Spatter formation in selective laser melting process using multi-laser technology[J]. Materials & Design,2017,131:460-469.
[15] LIU Y,YANG Y,MAI S,et al. Investigation into spatter behavior during selective laser melting of AISI 316L stainless steel powder[J]. Materials & Design,2015,87:797-806.
[16] GUNENTHIRAM V,PEYRE P,SCHNEIDER M,et al. Analysis of laser-melt pool-powder bed interaction during the selective laser melting of a stainless steel[J]. Journal of Laser Applications,2017,29(2):022303.
[17] MATTHEWS M J,GUSS G,KHAIRALLAH S A,et al. Denudation of metal powder layers in laser powder bed fusion processes[J]. Acta Materialia,2016,114:33-42.
[18] GRASSO M,DEMIR A G,PREVITALI B,et al. In situ monitoring of selective laser melting of zinc powder via infrared imaging of the process plume[J]. Robotics and Computer-Integrated Manufacturing,2018,49:229-239.
[19] ZHANG Y,FUH J Y H,YE D,et al. In-situ monitoring of laser-based PBF via off-axis vision and image processing approaches[J]. Additive Manufacturing,2019,25:263-274.
[20] REPOSSINI G,LAGUZZA V,GRASSO M,et al. On the use of spatter signature for in-situ monitoring of laser powder bed fusion[J]. Additive Manufacturing,2017,16:35-48.
[21] 王奉涛,杨守华,吕秉华,等. 金属增材制造过程熔池动态监测研究综述[J/OL]. 计算机集成制造系统:1-23[2022-04-30]. http://kns.cnki.net/kcms/detail/11.5946.TP.20211102.1345.004.html. WANG Fengtao,YANG Shouhua,LÜ Binghua,et al. Review of research on dynamic monitoring of metal additive manufacturing process[J/OL]. Computer Integrated Manufacturing Systems:1-23[2022-04-30]. http://kns.cnki.net/kcms/detail/11.5946.TP.20211102.1345.004.html.
[22] 曹龙超,周奇,韩远飞,等. 激光选区熔化增材制造缺陷智能监测与过程控制综述[J]. 航空学报,2021,42(10):199-233.
[23] CAO Longchao,ZHOU Qi,HAN Yuanfei,et al. Review on intelligent monitoring of defects and process control of selective laser melting additive manufacturing[J]. Acta Aeronautica et Astronautica Sinica,2021,42(10):199-233.
[24] TAHERI ANDANI M,DEHGHANI R,KARAMOOZ-RAVARI M R,et al. A study on the effect of energy input on spatter particles creation during selective laser melting process[J]. Additive Manufacturing,2018,20:33-43.
[25] 赵林君,张国庆,张大林,等. 基于最大熵阈值分割算法的激光选区熔化过程溅射特征研究[J/OL]. 激光与光电子学进展:1-12[2022-04-30]. http://kns.cnki.net/kcms/detail/31.1690.TN.20211116.1007.004.html. ZHAO Linjun,ZHANG Guoqing,ZHANG Dalin,et al. Investigation of spatter characteristics in SLM using the maximum entropy threshold segmentation algorithm[J/OL]. Laser & Optoelectronics Progress:1-12[2022-04-30]. http://kns.cnki.net/kcms/detail/31.1690.TN.20211116.1007.004.html.
[26] YANG D,LI H,LIU S,et al. In situ capture of spatter signature of SLM process using maximum entropy double threshold image processing method based on genetic algorithm[J]. Optics & Laser Technology,2020,131:106371.
[27] WANG D,DOU W,OU Y,et al. Characteristics of droplet spatter behavior and process-correlated mapping model in laser powder bed fusion[J]. Journal of Materials Research and Technology,2021,12:1051-1064.
[28] KWON O,KIM H G,HAM M J,et al. A deep neural network for classification of melt-pool images in metal additive manufacturing[J]. Journal of Intelligent Manufacturing,2020,31(2):375-386.
[29] YE D,FUH J Y H,ZHANG Y,et al. In situ monitoring of selective laser melting using plume and spatter signatures by deep belief networks[J]. ISA Transactions,2018,81:96-104.
[30] ZHANG Y,HONG G S,YE D,et al. Extraction and evaluation of melt pool,plume and spatter information for powder-bed fusion AM process monitoring[J]. Materials & Design,2018,156:458-469.
[31] ZHANG Y,SOON H G,YE D,et al. Powder-bed fusion process monitoring by machine vision with hybrid convolutional neural networks[J]. IEEE Transactions on Industrial Informatics,2020,16(9):5769-5779.
[32] TAN Z,FANG Q,LI H,et al. Neural network based image segmentation for spatter extraction during laser-based powder bed fusion processing[J]. Optics & Laser Technology,2020,130:106347.
[33] FANG Q,TAN Z,LI H,et al. In-situ capture of melt pool signature in selective laser melting using u-net-based convolutional neural network[J]. Journal of Manufacturing Processes, 2021,68:347-355.
[34] SCIME L,SIDDEL D,BAIRD S,et al. Layer-wise anomaly detection and classification for powder bed additive manufacturing processes:A machine-agnostic algorithm for real-time pixel-wise semantic segmentation[J]. Additive Manufacturing,2020,36:101453.
[35] MI J,ZHANG Y,LI H,et al. In-situ monitoring laser based directed energy deposition process with deep convolutional neural network[J]. Journal of Intelligent Manufacturing,2021:1-11.
[36] LIU W,ANGUELOV D,ERHAN D,et al. SSD:single shot multibox detector[C]//European Conference on Computer Vision,Springer,Cham,2016:21-37.
[37] REDMON J,DIVVALA S,GIRSHICK R,et al. You only look once:Unified,real-time object detection[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition,2016:779-788.
[38] REN S,HE K,GIRSHICK R,et al. Faster R-CNN:Towards real-time object detection with region proposal networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence,2017,39(6):1137-1149.
[39] 许德刚,王露,李凡. 深度学习的典型目标检测算法研究综述[J]. 计算机工程与应用,2021,57(8):10-25. XU Degang,WANG Lu,LI Fan. Review of typical object detection algorithms for deep learning[J]. Computer Engineering and Applications,2021,57(8):10-25.
[40] CHEN Q,WANG Y,YANG T,et al. You only look one-level feature[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2021:13039-13048.
[41] LIU J J,HOU Q,CHENG M M,et al. Improving convolutional networks with self-calibrated convolutions[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2020:10096-10105.
[42] WOO S,PARK J,LEE J Y,et al. CBAM:convolutional block attention module[C]//Proceedings of the European Conference on Computer Vision (ECCV). 2018:3-19.
[43] YE D,HONG G S,ZHANG Y,et al. Defect detection in selective laser melting technology by acoustic signals with deep belief networks[J]. The International Journal of Advanced Manufacturing Technology,2018,96(5):2791-801.

基于目标检测的选区激光熔融成形过程熔池与飞溅监测

Melt Pool and Spatter Monitoring in Selective Laser Melting Forming Process Based on Target Detection

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

本文评价

[1]	林昕, 毛杨坤, 傅盈西, 朱锟鹏. 基于图像语义分割与图卷积的选区激光熔融成形过程羽流运动特征分析[J]. 机械工程学报, 2024, 60(12): 168-182.
[2]	林晨, 何智成, 黄怡菲, 林智桂, 付广, 黄晋. 多级参数融合网络的驾驶场景目标检测方法研究[J]. 机械工程学报, 2024, 60(10): 64-75.
[3]	王亚辉, 郑联语, 樊伟, 赵雄, 张月红. 事件驱动考虑加工任务和监测状态的大型构件装配界面切削参数在线优化[J]. 机械工程学报, 2023, 59(24): 18-33.
[4]	彭湃, 耿可可, 王子威, 柳智超, 殷国栋. 智能汽车环境感知方法综述[J]. 机械工程学报, 2023, 59(20): 281-303.
[5]	刘永刚, 于丰宁, 章新杰, 陈峥, 秦大同. 基于激光点云与图像融合的3D目标检测研究[J]. 机械工程学报, 2022, 58(24): 289-299.
[6]	汪飞雪, 姚龙飞, 张天翊, 姚静, 张超, 孔祥东. 基于SLM工艺的变截面四棱锥点阵结构建模与试验研究[J]. 机械工程学报, 2021, 57(24): 158-165.
[7]	赵子婧, 刘宏哲, 曹东璞. 基于Libra R-CNN改进的交通标志检测算法[J]. 机械工程学报, 2021, 57(22): 255-265.
[8]	訾斌, 尹泽强, 李永昌, 赵涛. 基于YOLO模型的柔索并联机器人移动构件快速定位方法[J]. 机械工程学报, 2019, 55(3): 64-72.
[9]	吕杰, 罗芳颖, 袁泽剑. 目标搜索与识别的视觉注意网络与学习方法[J]. 机械工程学报, 2019, 55(11): 123-130.
[10]	赵飞;梅雪松;李光东;陶涛;姜歌东. 数控成型磨齿机加工误差在线监测及补偿[J]. , 2013, 49(1): 171-177.
[11]	高向东;丁度坤;赵传敏. 用于焊缝位置识别的视觉模型设计与试验[J]. , 2009, 45(4): 136-141.
[12]	岳建锋;李亮玉;范芳蕾;王天琪. 弧焊机器人熔化及气体保护焊熔池视觉自适应检测[J]. , 2008, 44(4): 206-210.
[13]	高向东;罗锡柱. 基于卡尔曼滤波的焊缝检测技术研究[J]. , 2004, 40(4): 172-176.
[14]	武传松;胡庆贤;孙俊生;Polte T.;Rehfeldt D.. 基于12维统计矢量的GMAW焊接过程监测模糊神经网络系统[J]. , 2004, 40(1): 151-154.
[15]	王建军;林涛;陈善本. 铝合金TIG焊熔池图像的获取与处理[J]. , 2003, 39(5): 125-129.