Multi-sensor Data Fusion Diagnosis Method Based on Interpretable Spatial-temporal Graph Convolutional Network

doi:10.3901/JME.2024.12.158

Abstract

Abstract: With the development of big data and artificial intelligence technologies, deep learning and its derived algorithms have achieved fruitful results and have been widely used in the field of fault diagnosis. However, intelligent fault diagnosis methods also face many challenges：1) existing multi-sensor data fusion methods have difficulty fully exploring the spatial-temporal information between multiple sensors to optimize diagnostic performance; 2) the decision-making process of intelligent diagnostic models has weak interpretability and cannot meet the reliability requirements in actual industrial scenarios. Therefore, we propose a multi-sensor data fusion diagnostic method based on an interpretable spatial-temporal graph convolutional network(ISTGCN). Firstly, a gate convolution layer is constructed to learn and enhance time features. Secondly, by combining the spatial layout relationship of sensors and the multi-sensor information fusion ability of graph convolutional networks, the spatial features of multi-sensor data are learned and extracted, and the effectiveness of the model is verified through the planetary gearbox fault diagnosis task. Finally, the improved gradient-based activation mapping algorithm is used to analyze the importance of each sensor data to the model diagnostic decision-making process, thereby improving the interpretability of the model update process. The experimental results show that the proposed method not only has good diagnostic performance but also provides an effective post-explanation method for multi-source sensor fusion diagnosis.

Key words: intelligent fault diagnosis, graph convolutional networks, explainable artificial intelligence, planetary gearboxes

CLC Number:

TG156

WEN Kairu, CHEN Zhuyun, HUANG Ruyi, LI Dongpeng, LI Weihua. Multi-sensor Data Fusion Diagnosis Method Based on Interpretable Spatial-temporal Graph Convolutional Network[J]. Journal of Mechanical Engineering, 2024, 60(12): 158-167.

References

[1] 黄如意，李霁蒲，王震，等. 基于多任务学习的装备智能诊断与寿命预测方法[J]. 中国科学：技术科学，2022，52：123-137. HUANG Ruyi，LI Jipu，WANG Zhen，et al. Intelligent diagnostic and prognostic method based on multitask learning for industrial equipment[J]. Journal of Mechanical Engineering，2022，52：123-137.
[2] LI W，HUANG R，LI J，et al. A perspective survey on deep transfer learning for fault diagnosis in industrial scenarios：Theories，applications and challenges[J]. Mechanical Systems and Signal Processing，2022，167：108487.
[3] 陈祝云，钟琪，黄如意，等. 基于增强迁移卷积神经网络的机械智能故障诊断[J]. 机械工程学报，2021，57(21)：96-105. CHEN Zhuyun，ZHONG Qi，HUANG Ruyi，et al. Intelligent fault diagnosis for machinery based on enhanced transfer convolutional neural network[J]. Journal of Mechanical Engineering，2021，57(21)：96-105.
[4] 孙文珺，邵思羽，严如强. 基于稀疏自动编码深度神经网络的感应电动机故障诊断[J]. 机械工程学报，2016，52(9)：65-71. SUN Wenjun，SHAO Siyu，YAN Ruqiang. Induction motor fault diagnosis based on deep neural network of sparse auto-encoder[J]. Journal of Mechanical Engineering，2016，52(9)：65-71.
[5] XIAO Y，SHAO H，HAN S Y，et al. Novel joint transfer network for unsupervised bearing fault diagnosis from simulation domain to experimental domain[J]. IEEE/ASME Transactions on Mechatronics，2022，27(6)：5254-5263.
[6] HE F，YE Q. A bearing fault diagnosis method based on wavelet packet transform and convolutional neural network optimized by simulated annealing algorithm[J]. Sensors，2022，22(4)：1410.
[7] 严如强，钱宇宁，胡世杰，等. 基于小波域平稳子空间分析的风力发电机齿轮箱故障诊断[J]. 机械工程学报，2014，50(11)：9-16. YAN Ruqiang，QIAN Yuning，HU Shijie，et al. Wind turbine gearbox fault diagnosis based on wavelet domain stationary subspace analysis[J]. Journal of Mechanical Engineering，2014，50(11)：9-16.
[8] 雷亚国，何正嘉，林京，等. 行星齿轮箱故障诊断技术的研究进展[J]. 机械工程学报，2011，47(19)：59-67. LEI Yaguo，HE Zhengjia，LIN Jing，et al. Research advances of fault diagnosis technique for planetary gearboxes [J]. Journal of Mechanical Engineering，2011，47(19)：59-67.
[9] JING L，WANG T，ZHAO M，et al. An adaptive multi-sensor data fusion method based on deep convolutional neural networks for fault diagnosis of planetary gearbox[J]. Sensors，2017，17(2)：414.
[10] DUAN Z，WU T，GUO S，et al. Development and trend of condition monitoring and fault diagnosis of multi-sensors information fusion for rolling bearings：a review[J]. The International Journal of Advanced Manufacturing Technology，2018，96：803-819.
[11] LIU Y，YAN X，ZHANG C，et al. An ensemble convolutional neural networks for bearing fault diagnosis using multi-sensor data[J]. Sensors，2019，19(23)：5300.
[12] GUAN Y，MENG Z，SUN D，et al. 2Mnet：multi-sensor and multi-scale model toward accurate fault diagnosis of rolling bearing[J]. Reliability Engineering & System Safety，2021，216：108017.
[13] WANG X，MAO D，LI X. Bearing fault diagnosis based on vibro-acoustic data fusion and 1D-CNN network[J]. Measurement，2021，173：108518.
[14] YANG C，LIU J，ZHOU K，et al. An improved multi-channel graph convolutional network and its applications for rotating machinery diagnosis[J]. Measurement，2022，190：110720.
[15] BASAK S，SENGUPTA S，WEN S J，et al. Spatio-temporal AI inference engine for estimating hard disk reliability[J]. Pervasive and Mobile Computing，2021，70：101283.
[16] YANG B，LEI Y，JIA F，et al. A polynomial kernel induced distance metric to improve deep transfer learning for fault diagnosis of machines[J]. IEEE Transactions on Industrial Electronics，2019，67(11)：9747-9757.
[17] SELVARAJU R R，COGSWELL M，DAS A，et al. Grad-cam：visual explanations from deep networks via gradient-based localization[J]. International Journal of Computer Vision，2019，128(2)：336-359.
[18] POPE P E，KOLOURI S，ROSTAMI M，et al. Explainability methods for graph convolutional neural networks[J]. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)，2019，10772-10781.
[19] CHEN Z，XU J，PENG T，et al. GCN-CAM：A new graph convolutional network-based fault diagnosis method with its interpretability analysis[C]// SAFEPROCESS，2021：1-6.
[20] KIPF T N，WELLING M. Semi-supervised classification with graph convolutional networks[J]. arXiv preprint arXiv，2016：1609.02907.
[21] YU B，YIN H，ZHU Z. Spatio-temporal graph convolutional networks：A deep learning framework for traffic forecasting[J]. arXiv preprint arXiv，2017：1709.04875.
[22] VELICKOVIC P，CUCURULL G，CASANOVA A，et al. Graph attention networks[J]. arXiv preprint arXiv，2017：1710.10903.