基于图像语义分割与图卷积的选区激光熔融成形过程羽流运动特征分析

doi:10.3901/JME.2024.12.168

机械工程学报 ›› 2024, Vol. 60 ›› Issue (12): 168-182.doi: 10.3901/JME.2024.12.168

• 特邀专栏：可解释可信AI驱动的智能监测与诊断 • 上一篇下一篇

扫码分享

基于图像语义分割与图卷积的选区激光熔融成形过程羽流运动特征分析

林昕^1,2, 毛杨坤³, 傅盈西⁴, 朱锟鹏^1,3

1. 武汉科技大学精密制造研究院武汉 430081;
2. 武汉科技大学冶金装备及其控制教育部重点实验室武汉 430081;
3. 中国科学院合肥物质科学研究院智能机械研究所合肥 230031;
4. 新加坡国立大学新国大苏州研究院苏州 215123

收稿日期:2023-08-01 修回日期:2024-03-06 出版日期:2024-06-20 发布日期:2024-08-23
作者简介:林昕,女,1987年出生,教授,博士研究生导师。主要从事智能制造、3D打印等在线/原位监测系统理论与技术实现的研究。E-mail:xinlin@wust.edu.cn;朱锟鹏(通信作者),男,1977年出生,博士,研究员,博士研究生导师。主要从事精密制造与自动化相关领域的理论和应用研究。E-mail:zhukp@iamt.ac.cn
基金资助:
国家自然科学基金资助项目(52175481)。

Analysis of Plume Motion Characteristics in Selective Laser Melting Forming Process Based on Image Semantic Segmentation and Graph Convolution

LIN Xin^1,2, MAO Yangkun³, FUH Ying Hsi Jerry⁴, ZHU Kunpeng^1,3

1. Precision Manufacturing Institute, Wuhan University of Science and Technology, Wuhan 430081;
2. Key Laboratory of Metallurgical Equipment and Control Technology, Ministry of Education, Wuhan University of Science and Technology, Wuhan 430081;
3. Institute of Intelligent Machines, Hefei Institutes of Physical Science, Chinese Academy of Sciences, Hefei 230031;
4. National University of Singapore (Suzhou) Research Institute, Suzhou 215123

Received:2023-08-01 Revised:2024-03-06 Online:2024-06-20 Published:2024-08-23

摘要/Abstract

摘要： 现有的研究中羽流已被证明与选区激光熔融加工过程有着密切关系且影响了最终成形质量，如何利用羽流等监测信息揭示选区激光熔融成形过程中熔融状态，是选区激光熔融成形过程监测被广泛关注的重要原因之一。为了更加完善地描述羽流信息，采用Deeplabv3+网络对图像进行语义分割，并使用标记图法建立一维的羽流形态特征；同时结合时序信号，构建图卷积自编码器与分类器，提出一种基于图结构的羽流运动特征，并进行聚类，得到熔融状态分类，讨论加工过程中羽流运动特征和熔融状态的关系。采用图卷积编码羽流动态特征，实现特征建立与熔融状态识别，并通过输出图卷积表征值，对图卷积在羽流特征提取中的应用进行可解释性分析。结果表明，提出的羽流特征能够实现五类熔融状态的识别且准确率达到了81.52%，并通过可解释性分析了羽流动态特征与熔融状态的内在关系，为选区激光熔融成形过程中监测信号与成形质量映射关系研究提供了新的思路。

关键词: 选区激光熔融, 图卷积, 过程监测, 羽流运动特征, 深度学习

Abstract: Plume is one of the important dynamic characteristics in the process of selective laser melting . How to use plume information to study the process of selective laser melting is one of the directions that has been widely concerned in the monitoring of selective laser melting process. In order to better describe the plume information, Deeplabv3+ network is used to conduct semantic segmentation of the image to obtain the plume contour, and establish a one-dimensional plume morphological feature. At the same time, combining with the time sequence signal, a graphic structure of plume motion feature is proposed, and the melting state category is obtained by clustering. In this way, the change of plume state during processing is discussed. Graph convolution was used to encode the dynamic features of the plume to realize the feature establishment and melting state recognition. The application of graph convolution in plume feature extraction was analyzed with the output of image convolution representation features. Based on the graph convolution, the graph convolution autoencoder and classifier are built to realize the classification of melting states, and the interpretability analysis of the graph convolution is carried out. The results show that the proposed plume features have high application value in the recognition of melting states. Finally, the recognition accuracy of the five types of melting states reaches 81.52%. The interpretability analysis shows how the features of the plume characterize the melting state, which provides a new idea for the study of the plume in the selective laser melting process.

Key words: selective laser melting, graph convolution, process monitoring, plume morphology

中图分类号:

TG156

林昕, 毛杨坤, 傅盈西, 朱锟鹏. 基于图像语义分割与图卷积的选区激光熔融成形过程羽流运动特征分析[J]. 机械工程学报, 2024, 60(12): 168-182.

LIN Xin, MAO Yangkun, FUH Ying Hsi Jerry, ZHU Kunpeng. Analysis of Plume Motion Characteristics in Selective Laser Melting Forming Process Based on Image Semantic Segmentation and Graph Convolution[J]. Journal of Mechanical Engineering, 2024, 60(12): 168-182.

参考文献

[1] 张家莲，李发亮，张海军. 选区激光熔化技术制备金属材料研究进展[J]. 激光与光电子学进展，2019，56(10)：35-44. ZHANG Jialian，LI Faliang，ZHANG Haijun. Research progress on preparation of metallic materials by selective laser melting[J]. Laser & Optoelectronics Progress，2019，56(10)：35-44.
[2] ZHU K，FUH J Y H，LIN X. Metal-based additive manufacturing condition monitoring：A review on machine learning based approaches [J]. IEEE/ASME Transactions on Mechatronics，2022，27(5)：2495-2510.
[3] SAHA S，ROY S. Metallic dental implants wear mechanisms，materials，and manufacturing processes：A literature review [J]. Materials，2023，16(1)：161.
[4] HUANG S，WEI H，LI D. Additive manufacturing technologies in the oral implant clinic：A review of current applications and progress[J]. Frontiers in Bioengineering and Biotechnology，2023，11：1100155.
[5] ZHANG J，SONG B，WEI Q，et al. A review of selective laser melting of aluminum alloys：Processing，microstructure，property and developing trends [J]. Journal of Materials Science & Technology，2019，35(2)：270-284.
[6] 杨益，党明珠，李伟，等. 激光选区熔化钛铝合金裂纹形成机理及抑制研究[J]. 机械工程学报，2020，56(3)：181-188. YANG Yi，DANG Mingzhu，LI Wei. Study on cracking mechanism and inhibiting process of TiAl alloys fabricated by selective laser melting[J]. Journal of Mechanical Engineering，2020，56(3)：181-188.
[7] 林鑫，黄卫东. 应用于航空领域的金属高性能增材制造技术[J]. 中国材料进展，2015，34(9)：684-688，658. LIN Xin，HUANG Weidong. High performance metal additive manufacturing technology applied in aviation field[J]. Materials China，2015，34(9)：684-688，658.
[8] 杨永强，陈杰，宋长辉，等.金属零件激光选区熔化技术的现状及进展[J]. 激光与光电子学进展，2018，55(1)：9-21. YANG Yongqiang，CHEN Jie，SONG Changhui，et al. Current status and progress on technology of selective laser melting of metal parts[J]. Laser & Optoelectronics Progress，2018，55(1)：9-21.
[9] ZHAO C，PARAB N D，LI X，et al. Critical instability at moving keyhole tip generates porosity in laser melting[J]. Science，2020，370(6520)：1080-1086.
[10] CHEN L，YAO X，XU P，et al. Rapid surface defect identification for additive manufacturing with in-situ point cloud processing and machine learning[J]. Virtual and Physical Prototyping，2021，16(1)：50-67.
[11] SHARMA S，JOSHI S S，PANTAWANE M V，et al. Multiphysics multi-scale computational framework for linking process-structure-property relationships in metal additive manufacturing：A critical review[J]. International Materials Reviews，2023，68(7)：1-67.
[12] 赵沧，杨源祺，师博，等. 金属激光增材制造微观结构和缺陷原位实时监测[J]. 科学通报，2022，67(25)：3036-3053. CANG Zhao，YANG Yuanqi，SHI Bo，et al. Operando monitoring microstructures and defects in laser fusion additive manufacturing of metals[J]. Chinese Science Bulletin，2022，67(25)：3036-3053.
[13] YANG T，JIN Y，SQUIRES B，et al. In-situ monitoring and ex-situ elasticity mapping of laser induced metal melting pool using ultrasound：Numerical and experimental approaches[J]. Journal of Manufacturing Processes，2021，71：178-186.
[14] MCCANN R，OBEIDI M A，HUGHES C，et al. In-situ sensing，process monitoring and machine control in laser powder bed fusion：A review[J]. Additive Manufacturing，2021，45：102058.
[15] WU Bo，JI Xiaoyuan，ZHOU Jianxin,et al. In situ monitoring methods for selective laser melting additive manufacturing process based on images-A review[J]. China Foundry，2021，18(4)：265-285.
[16] LIN X，ZHU K，FUH J Y H，et al. Metal-based additive manufacturing condition monitoring methods：From measurement to control[J]. ISA transactions，2022，120：147-166.
[17] LUO S Y，MA X Q，XU J，et al. Deep learning based monitoring of spatter behavior by the acoustic signal in selective laser melting[J]. Sensors，2021，21(21)：7179.
[18] YANG D，LI H，LIU S，et al. In situ capture of spatter signature of SLM process using maximum entropy double threshold image processing method based on genetic algorithm[J]. Optics & Laser Technology，2020，131：106371.
[19] TIAN Q，GUO S H，MELDER E，et al. Deep learning-based data fusion method for in situ porosity detection in laser-based additive manufacturing[J]. Journal of Manufacturing Science and Engineering，2021，143(4)：041011.
[20] LANE B，YEUNG H. Process monitoring dataset from the additive manufacturing metrology testbed (AMMT):“Three-dimensional scan strategies”[J]. Journal of Research of the National Institute of Standards and Technology，2019，124：1.
[21] HEIGEL J C，LANE B，LEVINE L，et al. In situ thermography of the metal bridge structures fabricated for the 2018 additive manufacturing benchmark test series (AM-Bench 2018)[J]. Journal of Research of the National Institute of Standards and Technology，2020，125：125005.
[22] LEVINE L，LANE B，HEIGEL J，et al. Outcomes and conclusions from the 2018 AM-Bench measurements，challenge problems，modeling submissions，and conference [J]. Integrating Materials and Manufacturing Innovation，2020，9(1)：1-15.
[23] LIN X，WANG Q S，FUH J Y H，et al. Motion feature based melt pool monitoring for selective laser melting process[J]. Journal of Materials Processing Technology，2022，303：117523.
[24] 朱锟鹏，王齐胜，林昕，等. 基于熔池运动特征的选区激光熔融过程状态检测方法[J]. 计算机集成制造系统，2021，27(12)：3403-3415. ZHU Kunpeng，WANG Qisheng，LIN Xin，et al. Selective laser melting process state monitoring method based on motion feature of melt pool[J]. Computer Integrated Manufacturing Systems，2021，27(12)：3403-3415.
[25] ZHANG Z，SAHU C K，SINGH S K，et al. Machine learning based prediction of melt pool morphology in a laser-based powder bed fusion additive manufacturing process[J]. International Journal of Production Research，2024，62(5)：1803-1817.
[26] COX M E，SCHWALBACH E J，BLAISZIK B J，et al. AFRL additive manufacturing modeling challenge series：Overview[J]. Integrating Materials and Manufacturing Innovation，2021，10(2)：125-128.
[27] 毛杨坤，段现银，林昕，等. 基于目标检测的选区激光熔融成形过程熔池与飞溅监测[J]. 机械工程学报，2023，59(9)：335-348. MAO Yangkun，DUAN Xianyin，LIN Xin，et al. Melt pool and spatter monitoring in selective laser melting forming process based on target detection[J]. Journal of Mechanical Engineering，2023，59(9)：335-348.
[28] GRASSO M，DEMIR A G，PREVITALI B，et al. In situ monitoring of selective laser melting of zinc powder via infrared imaging of the process plume [J]. Robotics and Computer-Integrated Manufacturing，2018，49：229-239.
[29] ZHANG Y，FUH J Y H，YE D，et al. In-situ monitoring of laser-based PBF via off-axis vision and image processing approaches [J]. Additive Manufacturing，2019，25：263-274.
[30] REPOSSINI G，LAGUZZA V，GRASSO M，et al. On the use of spatter signature for in-situ monitoring of laser powder bed fusion [J]. Additive Manufacturing，2017，16：35-48.
[31] YE D S，ZHU K P，FUH J Y H，et al. The investigation of plume and spatter signatures on melted states in selective laser melting [J]. Optics and Laser Technology，2019，111：395-406.
[32] GAO X D，SUN Y，KATAYAMA S. Neural network of plume and spatter for monitoring high-power disk laser welding[J]. International Journal of Precision Engineering and Manufacturing-Green Technology，2014，1(4)：293-298.
[33] CHEN X，PANG S Y，SHAO X Y，et al. Sub-microsecond vapor plume dynamics under different keyhole penetration regimes in deep penetration laser welding [J]. Journal of Physics D-Applied Physics，2017，50(20)：1-13.
[34] TAN P，KIRAN R，ZHOU K J J O M P. Effects of sub-atmospheric pressure on keyhole dynamics and porosity in products fabricated by selective laser melting [J]. Journal of Manufacturing Processes，2021，64：816-827.
[35] PODRABINNIK P A，SHTANKO A E，KHMYROV R S，et al. Interferometry of gas-phase flows during selective laser melting[J]. Applied Sciences，2019，10(1)：231.
[36] GRIGGS D A，GIBBS J S，BAKER S P，et al. A testbed for investigation of laser powder bed fusion at elevated atmospheric pressure[J]. Additive Manufacturing，2022，51：102581.
[37] WANG C M，MENG X X，HUANG W，et al. Role of side assisting gas on plasma and energy transmission during CO2 laser welding[J]. Journal of Materials Processing Technology，2011，211(4)：668-674.
[38] YIN J，HAO L，YANG L，et al. Investigation of interaction between vapor plume and spatter during selective laser melting additive manufacturing[J]. Chin. J. Lasers，2022，49：1402202.
[39] BIDARE P，BITHARAS I，WARD R M，et al. Fluid and particle dynamics in laser powder bed fusion [J]. Acta Materialia，2018，142：107-120.
[40] ZHANG Y J，YAN W T，PENG X J，et al. Investigation into the optical emission of features for powder-bed fusion AM process monitoring [J]. International Journal of Advanced Manufacturing Technology，2022，121(3-4)：2291-2303.
[41] YE D，HSI FUH J Y，ZHANG Y，et al. In situ monitoring of selective laser melting using plume and spatter signatures by deep belief networks [J]. ISA Transactions，2018，81：96-104.
[42] TAN Z，FANG Q，LI H，et al. Neural network based image segmentation for spatter extraction during laser-based powder bed fusion processing [J]. Optics & Laser Technology，2020，130：106347.
[43] FANG Q，TAN Z，LI H，et al. In-situ capture of melt pool signature in selective laser melting using U-Net-based convolutional neural network [J]. Journal of Manufacturing Processes，2021，68：347-355.
[44] MI J，ZHANG Y，LI H，et al. In-situ monitoring laser based directed energy deposition process with deep convolutional neural network[J]. Journal of Intelligent Manufacturing，2021，34(2)：683-693.
[45] ESCHNER E，STAUDT T，SCHMIDT M. Sensing approach for the in-situ determination of spatter motion within PBF-LB/M [J]. Cirp Annals-Manufacturing Technology，2022，71(1)：149-152.
[46] LI J C，CAO L C，XU J，et al. In situ porosity intelligent classification of selective laser melting based on coaxial monitoring and image processing[J]. Measurement，2022，187：110232.
[47] ZHANG Y，HONG G S，YE D，et al. Extraction and evaluation of melt pool，plume and spatter information for powder-bed fusion AM process monitoring [J]. Materials & Design，2018，156：458-469.
[48] YU B，YIN H，ZHU Z. Spatio-temporal graph convolutional networks：A deep learning framework for traffic forecasting [C]// Proceedings of the 27th International Joint Conference on Artificial Intelligence，2018：3634-3640.
[49] YAN S，XIONG Y，LIN D. Spatial temporal graph convolutional networks for skeleton-based action recognition[C/CD]//Proceedings of the AAAI Conference on Artificial Intelligence，2018.
[50] GUO W G，TIAN Q，GUO S，et al. A physics-driven deep learning model for process-porosity causal relationship and porosity prediction with interpretability in laser metal deposition [J]. CIRP Annals，2020，69(1)：205-208.
[51] ZHU Q，LIU Z，YAN J. Machine learning for metal additive manufacturing：predicting temperature and melt pool fluid dynamics using physics-informed neural networks [J]. Computational Mechanics，2021，67(2)：619-635.
[52] REN W，WEN G，ZHANG Z，et al. Quality monitoring in additive manufacturing using emission spectroscopy and unsupervised deep learning[J]. Materials and Manufacturing Processes，2022，37(11)：1339-1346.
[53] GUO H，ZHANG Y，ZHU K. Interpretable deep learning approach for tool wear monitoring in high-speed milling[J]. Computers in Industry，2022，138：103638.
[54] 蒲天骄，乔骥，赵紫璇，等. 面向电力系统智能分析的机器学习可解释性方法研究(一)：基本概念与框架[J].中国电机工程学报，2023，43(18)：7010-7029. PU Tianjiao，QIAO Ji，ZHAO Zixuan，et al. Research on interpretable methods of machine learning applied in intelligent analysis of power system (Part I)：Basic concept and framework[J]. Proceedings of the CSEE，2023，43(18)：7010-7029.
[55] CHEN L C E，ZHU Y K，PAPANDREOU G，et al. Encoder-decoder with atrous separable convolution for semantic image segmentation[C/CD]//Proceedings of the 15th European Conference on Computer Vision (ECCV)，Munich，Germany，Sep，8-14，2018.
[56] SANDLER M，HOWARD A，ZHU M，et al. Inverted residuals and linear bottlenecks：Mobile networks for classification，detection and segmentation [J/OL]//arXiv，https://doi.org/10.48550/arXiv.1801.04381.
[57] JIANG M，DONG J，MA D，et al. Inception spatial temporal graph convolutional networks for skeleton-based action recognition [C]//2022 International Symposium on Control Engineering and Robotics (ISCER)，2022：208-213.
[58] YU B，YIN H T，ZHU Z X. Spatio-temporal graph convolutional networks：A deep learning framework for traffic forecasting[C/CD]//Proceedings of the 27th International Joint Conference on Artificial Intelligence (IJCAI)，Stockholm，Sweden，Jul，13-19，2018.

基于图像语义分割与图卷积的选区激光熔融成形过程羽流运动特征分析

Analysis of Plume Motion Characteristics in Selective Laser Melting Forming Process Based on Image Semantic Segmentation and Graph Convolution

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

本文评价

[1]	李春凯, 王嘉昕, 石玗, 代悦. GTAW熔池激光条纹动态行为与熔透预测方法研究[J]. 机械工程学报, 2024, 60(6): 236-244.
[2]	赵德望, 姜超, 赵延广, 杨文平, 樊俊铃. 铝锂合金疲劳性能及预测研究-基于试验和“浅层”+“深度”混合神经网络方法[J]. 机械工程学报, 2024, 60(16): 190-199.
[3]	付玲, 佘玲娟, 颜镀镭, 张鹏, 龙湘云. 基于内嵌物理信息与注意力机制BiLSTM神经网络的臂架系统疲劳损伤预测模型[J]. 机械工程学报, 2024, 60(13): 205-215.
[4]	陈钱, 陈康康, 董兴建, 皇甫一樊, 彭志科, 孟光. 一种面向机械设备故障诊断的可解释卷积神经网络[J]. 机械工程学报, 2024, 60(12): 65-76.
[5]	刘岳开, 王天杨, 褚福磊. 基于低延迟可解释性深度学习的复杂旋转机械关键部件知识嵌入与诊断方法研究[J]. 机械工程学报, 2024, 60(12): 107-115.
[6]	赖旭伟, 丁昆, 张楷, 黄锋飞, 郑庆, 李致萱, 丁国富. 基于可解释物理引导空间注意力改进的跨工艺参数立铣刀磨损辨识[J]. 机械工程学报, 2024, 60(12): 147-157.
[7]	温楷儒, 陈祝云, 黄如意, 李东鹏, 李巍华. 基于可解释时空图卷积网络的多传感数据融合诊断方法[J]. 机械工程学报, 2024, 60(12): 158-167.
[8]	邵海东, 肖一鸣, 邓乾旺, 任颖莹, 韩特. 基于不确定性感知网络的可信机械故障诊断[J]. 机械工程学报, 2024, 60(12): 194-206.
[9]	李玖法, 邹博文, 任玥. 考虑车-路交互作用的车辆轨迹预测算法研究[J]. 机械工程学报, 2024, 60(10): 76-85.
[10]	许思远, 张卫东, 毛玉明, 牛斌. 数据驱动的指定变形非均质多胞结构优化设计[J]. 机械工程学报, 2024, 60(1): 85-95.
[11]	毛杨坤, 段现银, 林昕, 傅盈西, 朱锟鹏. 基于目标检测的选区激光熔融成形过程熔池与飞溅监测[J]. 机械工程学报, 2023, 59(9): 335-348.
[12]	何赟泽, 李响, 王洪金, 侯岳骏, 张帆, 牟欣颖, 刘浩, 程豪, 李时华, 李杰. 基于可见光和热成像的风机叶片全周期无损检测综述[J]. 机械工程学报, 2023, 59(6): 32-45.
[13]	王亚辉, 郑联语, 樊伟, 赵雄, 张月红. 事件驱动考虑加工任务和监测状态的大型构件装配界面切削参数在线优化[J]. 机械工程学报, 2023, 59(24): 18-33.
[14]	康杰, 王寅, 罗杰, 孙嘉宝, 曾舒洪. 自主工作模态分析方法研究综述[J]. 机械工程学报, 2023, 59(13): 89-109.
[15]	吴强威, 文龙. 基于联合对抗训练的深度学习表面缺陷检测方法[J]. 机械工程学报, 2023, 59(12): 173-182.