集成制造在线故障监测的多模态物理信息融合方法

doi:10.3901/JME.260825

机械工程学报 ›› 2026, Vol. 62 ›› Issue (15): 142-153.doi: 10.3901/JME.260825

• 特邀专栏：大模型时代的智能设计与制造 • 上一篇

扫码分享

集成制造在线故障监测的多模态物理信息融合方法

吕锐婵^1,2, 徐涵^1,2, 訾斌^1,2, 张培榑^1,2, 李宇吉^1,2, 黄进^1,2

1. 西安电子科技大学高性能电子装备机电集成制造全国重点实验室西安 710071;
2. 西安电子科技大学机电工程学院西安 710071

收稿日期:2025-08-15 修回日期:2025-11-19 发布日期:2026-10-13
作者简介:吕锐婵，女，1987年出生，博士，教授。主要研究方向为集成制造，光电传感与智能检测。E-mail：rclv@xidian.edu.cn;徐涵，男，2001年出生，博士研究生。主要研究方向为智能故障诊断。E-mail：xidianxuhan@163.com;訾斌(通信作者)，男，1975年出生，博士，教授。主要研究方向为刚柔耦合智能机器人，智能制造系统控制与自动化。E-mail：zibinhfut@163.com
基金资助:
国家自然科学基金(82472104和U24B2053)、陕西省自然科学基础研究(2025JC-JCQN-023)和陕西省关键核心技术研发(2024QY2-GJHX-03)资助项目。

Physical Information Fusion of Multi-modal Data for Online Fault Monitoring of Integrated Manufacturing

LÜ Ruichan^1,2, XU Han^1,2, ZI Bin^1,2, ZHANG Peifu^1,2, LI Yuji^1,2, HUANG Jing^1,2

1. State Key Laboratory of Electromechanical Integrated Manufacturing of High-performance Electronic Equipment, Xidian University, Xi'an 710071;
2. School of Electro-Mechanical Engineering, Xidian University, Xi'an 710071

Received:2025-08-15 Revised:2025-11-19 Published:2026-10-13

摘要/Abstract

摘要： 高性能电子装备集成制造(Integrated manufacturing, IM)技术是智能制造的核心关键。IM制造过程的稳定性和可靠性依赖于离线检测，往往导致构件缺陷发现不及时、故障过程不能实时监控，使得人机物浪费及停工停产带来的潜在损失。智能故障监测技术迫在眉睫，然而传统的多模态故障诊断算法缺乏物理信息融合以及上下文建模能力，难以充分挖掘信号的空间和非线性动力学特征。为了解决以上问题，提出了多模态物理信息融合网络(Multimodal physical information fusion network, MPIF-Net)用于选区激光熔融IM-SLM集成制造的在线故障监测。首先，通过多传感器部署进行多模态声发射数据采集；在保留原始的时序模态的同时将其转换为带有空间和非线性动力学特征的融合图像。其次，设计了双注意力卷积和Mamba block来捕获融合图像的局部和全局信息。与此同时，引入了BiLSTM架构来深入分析原始声发射信号的时序特征。特别的，构建了一个物理信息神经网络PINN来生成高保真的温度模态信号以提升故障监测的准确率。最后，通过通道拼接的融合策略实现了物理信息和数据的多模态融合。试验结果表明，MPIF-Net在多项监测指标上显著优于现有方法，F1指标较传统多模态算法提升1.48%。研究表明，该方法不仅有效提升了故障监测的准确性和可靠性，而且为未来集成制造领域的智能监测提供了可推广的解决方案。

关键词: 故障监测, 选区激光熔融, 集成制造, 多模态物理信息融合

Abstract: High-performance electronic equipment integrated manufacturing (IM) technology is essential for intelligent manufacturing. The stability and reliability of IM’ s monitoring are dependent on the offline detection after manufacturing, which make slate detection of the products and manufacturing process, resulting in the waste of human-machine-time and the potential loss caused by the stop work and production. Intelligent fault monitoring is essential, while lacking physical information fusion and contextual modeling capability and facing multiple challenges like difficult fully exploiting spatial and nonlinear dynamic features of signals. To address these limitations, a multimodal physical information fusion network (MPIF-Net) is proposed for online fault monitoring in IM-SLM processes. Multi-sensor deployment is employed to collect multimodal acoustic emission data, which are transformed into fused images containing spatial and nonlinear dynamic features while retaining the original temporal modality. A dual-attention convolution module and Mamba block are designed to capture both local and global information from the fused images. In parallel, a BiLSTM architecture is incorporated to extract temporal dependencies from the raw acoustic emission signals. In particular, a physics-informed neural network (PINN) is constructed to generate high-fidelity thermal modality signals, thereby improving monitoring accuracy. Finally, a channel-concatenation fusion strategy integrates physical information with data-driven features to achieve multimodal fusion. Experimental results demonstrate that MPIF-Net significantly outperforms existing methods across multiple monitoring metrics, achieving an improvement of 1.48% in the F1 score compared with conventional multimodal algorithms. The findings indicate that the method not only enhances the accuracy and reliability of fault monitoring but also provides a generalizable solution for intelligent monitoring in integrated manufacturing.

Key words: fault monitoring, selective laser melting, integrated manufacturing, multi-modal physical information fusion

中图分类号:

TG156

吕锐婵, 徐涵, 訾斌, 张培榑, 李宇吉, 黄进. 集成制造在线故障监测的多模态物理信息融合方法[J]. 机械工程学报, 2026, 62(15): 142-153.

LÜ Ruichan, XU Han, ZI Bin, ZHANG Peifu, LI Yuji, HUANG Jing. Physical Information Fusion of Multi-modal Data for Online Fault Monitoring of Integrated Manufacturing[J]. Journal of Mechanical Engineering, 2026, 62(15): 142-153.

导出引用管理器 EndNote|Reference Manager|ProCite|BibTeX|RefWorks

链接本文: http://www.cjmenet.com.cn/CN/10.3901/JME.260825

http://www.cjmenet.com.cn/CN/Y2026/V62/I15/142

参考文献

[1] 蒋周明矩，熊异，王柏村. 面向工业5.0的人机协作增材制造[J]. 机械工程学报，2024，60(3)：238-253. JIANG Zhoumingju，XIONG Yi，WANG Baicun，et al. Human-machine collaborative additive manufacturing for industry 5.0[J]. Journal of Mechanical Engineering，2024，60(3)：238-253.
[2] FUNCH C V，PROUST G. Laser-based additive manufacturing of refractory metals and their alloys：A review[J]. Additive Manufacturing，2024，94：104464.
[3] ZHOU C，YAN X，WANG H，et al. Advancements in hydrogen embrittlement of selective laser melting austenitic stainless steel：Mechanisms，microstructures，and future directions[J]. Journal of Materials Science & Technology，2025，230：219-235.
[4] YIN Z，HOU J. Recent advances on SVM based fault diagnosis and process monitoring in complicated industrial processes[J]. Neurocomputing，2016，174：643-650.
[5] KHANZADEH M，TIAN W，YADOLLAHI A，et al. Dual process monitoring of metal-based additive manufacturing using tensor decomposition of thermal image streams[J]. Additive Manufacturing，2018，23：443-456.
[6] CHALESHTORI A E，AGHAIE A. A novel bearing fault diagnosis approach using the Gaussian mixture model and the weighted principal component analysis[J]. Reliability Engineering & System Safety，2024，242：109720.
[7] 雷亚国，贾峰，孔德同，等. 大数据下机械智能故障诊断的机遇与挑战[J]. 机械工程学报，2018，54(5)：94-104. LEI Yanguo，JIA Feng，KONG Detong，et al. Opportunities and challenges of machinery intelligent fault diagnosis in big data era[J]. Journal of Mechanical Engineering，2018，54(5)：94-104.
[8] 严如强，许文纲，王志颖，等. 航空发动机燃油控制系统故障诊断技术研究进展与挑战[J]. 机械工程学报，2024，60(4)：3-31. YAN Ruqiang，XU Wengang，WANG Zhiying，et al. Research status and challenges on fault diagnosis methodology for fuel control system of aero-engine[J]. Journal of Mechanical Engineering，2024，60(4)：3-31.
[9] 狄子钧，袁东风，李东阳，等. 基于多尺度-高效通道注意力网络的刀具故障诊断方法[J]. 机械工程学报，2024，60(6)：82-90. DI Zijun，YUAN Dongfeng，LI Dongyang，et al. Tool fault diagnosis method based on multiscale-efficient channel attention network[J]. Journal of Mechanical Engineering，2024，60(6)：82-90.
[10] XIE T，HUANG X，CHOI S K. Intelligent mechanical fault diagnosis using multisensor fusion and convolution neural network[J]. IEEE Transactions on Industrial Informatics，2021，18(5)：3213-3223.
[11] PU Y，ZHAO Y，ZHAO G，et al. Experimental study on laser assisted machining of silicon nitride ceramics based on acoustic emission detection[J]. Optics & Laser Technology，2025，180：111497.
[12] DU J，ZENG J，CHEN C，et al. Acoustic emission monitoring for damage diagnosis in composite laminates based on deep learning with attention mechanism[J]. Mechanical Systems and Signal Processing，2025，222：111770.
[13] FU S，ZOU L，WANG Y，et al. DCSIAN：A novel deep cross-scale interactive attention network for fault diagnosis of aviation hydraulic pumps and generalizable applications[J]. Reliability Engineering & System Safety，2024，249：110246.
[14] ZHANG S，LIANG W，ZHAO W，et al. Electro-hydraulic SBW fault diagnosis method based on novel 1DCNN-LSTM with attention mechanisms and transfer learning[J]. Mechanical Systems and Signal Processing，2024，220：111644.
[15] 温楷儒，陈祝云，黄如意，等. 基于可解释时空图卷积网络的多传感数据融合诊断方法[J]. 机械工程学报，2024，60(12)：158-167. WEN Kairu，CHEN Zhuyun，HUANG Ruyi，et al. Multi-sensor data fusion diagnosis method based on interpretable spatial-temporal graph convolutional network[J]. Journal of Mechanical Engineering，2024，60(12)：158-167.
[16] 陈其怀，林添良，马荣华，等. 基于多模态的履带式挖掘机自动驾驶决策网络[J]. 机械工程学报，2025，61(4)：380-388. CHEN Qihuai，LIN Tianliang，MA Ronghua，et al. Automatic driving decision network of crawler excavator based on multi-modal[J]. Journal of Mechanical Engineering，2025，61(4)：380-388.
[17] PETRICH J，SNOW Z，CORBIN D，et al. Multi-modal sensor fusion with machine learning for data-driven process monitoring for additive manufacturing[J]. Additive Manufacturing，2021，48：102364.
[18] XU H，LV R，ZI B. A multi-modal information fusion network combining CNN and adaptive Mamba for fault monitoring of selective laser melting ceramic printing instrument[J/OL]. IEEE Transactions on Instrumentation and Measurement，2025，doi：10.1109/TIM.2025.3585221.
[19] KHAN A，SOHAIL A，ZAHOORA U，et al. A survey of the recent architectures of deep convolutional neural networks[J]. Artificial Intelligence Review，2020，53：5455-5516.
[20] GU A，GOEL K，RÉ C. Efficiently modeling long sequences with structured state spaces[J]. arXiv preprint arXiv：2111.00396，2021.
[21] HOCHREITER S，SCHMIDHUBER J. Long short-term memory[J]. Neural Computation，1997，9(8)：1735-1780.
[22] WOO S，PARK J，LEE J-Y，et al. Cbam：Convolutional block attention module[C]//Proceedings of the European Conference on Computer Vision (ECCV)，2018：3-19.
[23] DOSOVITSKIY A，BEYER L，KOLESNIKOV A，et al. An image is worth 16x16 words：Transformers for image recognition at scale[J]. arXiv preprint arXiv：2010.11929，2020.
[24] ZHANG W，PENG G，LI C，et al. A new deep learning model for fault diagnosis with good anti-noise and domain adaptation ability on raw vibration signals[J]. Sensors，2017，17(2)：425.
[25] WANG W，LI X，XU Z，et al. Learning two-stream CNN for multi-modal age-related macular degeneration categorization[J]. IEEE Journal of Biomedical and Health Informatics，2022，26(8)：4111-4122.
[26] WANG L，ZHAO W. An ensemble deep learning network based on 2D convolutional neural network and 1D LSTM with self-attention for bearing fault diagnosis[J]. Applied Soft Computing，2025，172：112889.
[27] XIE J R，SAFDAR M，CHEN L Q，et al. Audio-visual cross-modality knowledge transfer for machine learning- based in-situ monitoring in laser additive manufacturing[J]. Additive Manufacturing，2025，101：104692.

集成制造在线故障监测的多模态物理信息融合方法

Physical Information Fusion of Multi-modal Data for Online Fault Monitoring of Integrated Manufacturing

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 10

编辑推荐

Metrics

本文评价

[1]	林昕, 毛杨坤, 傅盈西, 朱锟鹏. 基于图像语义分割与图卷积的选区激光熔融成形过程羽流运动特征分析[J]. 机械工程学报, 2024, 60(12): 168-182.
[2]	毛杨坤, 段现银, 林昕, 傅盈西, 朱锟鹏. 基于目标检测的选区激光熔融成形过程熔池与飞溅监测[J]. 机械工程学报, 2023, 59(9): 335-348.
[3]	汪飞雪, 姚龙飞, 张天翊, 姚静, 张超, 孔祥东. 基于SLM工艺的变截面四棱锥点阵结构建模与试验研究[J]. 机械工程学报, 2021, 57(24): 158-165.
[4]	刘小平;徐桂云;任世锦;杨茂云. 基于HHT-DDKICA和支持矢量数据描述方法的提升机故障监测[J]. , 2012, 48(23): 65-75.
[5]	陈峙;王铁;谷丰收;MANSAF Haram;ANDREW Ball. 基于电动机电流信号双谱分析的齿轮传动故障诊断[J]. , 2012, 48(21): 84-90.
[6]	杨叔子;吴波;李斌. 再论先进制造技术及其发展趋势[J]. , 2006, 42(1): 1-5.
[7]	陈新;陈庆新. 中国的网络化制造模式及系统[J]. , 2003, 39(11): 74-81.
[8]	谢平;刘彬. 一种基于小波网络的切削刀具故障监测[J]. , 2002, 38(2): 108-111.
[9]	毕诸明;张友良. 一种CIMS体系结构研究的新构思[J]. , 1997, 33(5): 17-24.
[10]	董明;查建中. 基于知识的集成制造柔性仿真系统[J]. , 1996, 32(6): 77-83.