可解释人工智能在工业智能诊断中的挑战和机遇：归因解释

doi:10.3901/JME.2024.12.021

摘要/Abstract

摘要： 针对目前迅猛发展的工业智能诊断方法缺乏可解释性开展综述，指出模型无关的归因解释技术在工业智能诊断中的研究现状和潜在研究方向。分析可解释性技术的主要观点和作用，针对工业智能诊断的两个特性问题—非线性高维观测、知识表征精度低，归因解释技术可以提供有效的前向理解智能模型逻辑结构、反向优化模型设计的工具。从注意力机制、显著性分析、规则提取、代理模型四个方面，概述其主要观点与作用，介绍现有方法的研究现状，并总结分析不同归因解释技术的优势与不足。通过四个案例分析，阐述不同归因解释技术在智能诊断中的效果。最后展望归因解释技术在工业智能诊断中的研究方向，包括可解释性量化、反馈模型设计、模型复杂性与可解释性平衡、高维特征的归因解释，期望为可解释人工智能技术在工业智能诊断中的发展提供方向建议。

关键词: 工业智能诊断, 可解释性, 模型无关, 归因分析

Abstract: The purpose is to figure the lack of interpretability for current industrial intelligence diagnosis methods, review the development situation of model-agnostics attribution analysis in industrial intelligence diagnosis and point out the potential development direction. The main viewpoints and functions of interpretable techniques are analyzed. Aiming at two characteristic problems of industrial intelligence diagnosis, i.e., nonlinear high-dimensional observation and inaccurate knowledge representation, attribution interpretation provides effective methods for understanding forward logical structure and reverse optimizing design of intelligent models. The core concepts, existing works and pros and cons of attention mechanism, saliency analysis, rule extraction, and proxy model are systematically summarized. Four case studies are used to illustrate the result of attribution interpretation techniques. Finally, potential research directions of attribution interpretation technology in industrial intelligent diagnosis are discussed, including quantification of interpretability, feedback to model design, balance between model complexity and interpretability, and attribution analysis in high dimension. Through this review, we hope to provide a suggestion to conduct further development of interpretable intelligence in industrial fault diagnosis.

Key words: industrial intelligence diagnosis, interpretability, model-agnostic, attribution analysis

中图分类号:

TH17

严如强, 周峥, 杨远贵, 李亚松, 胡晨烨, 陶治宇, 赵志斌, 王诗彬, 陈雪峰. 可解释人工智能在工业智能诊断中的挑战和机遇：归因解释[J]. 机械工程学报, 2024, 60(12): 21-40.

YAN Ruqiang, ZHOU Zheng, YANG Yuangui, LI Yasong, HU Chenye, TAO Zhiyu, ZHAO Zhibin, WANG Shibing, CHEN Xuefeng. Challenges and Opportunities of XAI in Industrial Intelligent Diagnosis: Attribution Interpretation[J]. Journal of Mechanical Engineering, 2024, 60(12): 21-40.

参考文献

[1] 陈雪峰，李继猛，程航，等. 风力发电机状态监测和故障诊断技术的研究与进展[J]. 机械工程学报，2011，47(9)：45-52. CHEN Xuefeng，LI Jimeng，CHENG Hang，et al. Research and application of condition monitoring and fault diagnosis technology in wind turbines[J]. Journal of Mechanical Engineering，2011，47(9)：45-52.
[2] 沈田，刘宗阳，李豪，等. 结合门控循环单元的轴承故障声发射信息表征机制与定位[J/OL]. 振动工程学报，[2023-03-29]. http://kns.cnki.net/kcms/detail/32.1349.TB.20230119.1315.003.html. SHEN Tian，LIU Zongyang，LI Hao，et al. Characterization and localization mechanism of bearing fault acoustic emission information based on gated recurrent unit[J/OL]. Journal of Vibration Engineering, [2023-03-29]. http://kns.cnki.net/kcms/detail/32.1349.TB.20230119.1315.003.html.
[3] 韩特，李彦夫，雷亚国，等. 融合图标签传播和判别特征增强的工业机器人关键部件半监督故障诊断方法[J]. 机械工程学报，2022，58(17)：116-124. HAN Te，LI Yanfu，LEI Yaguo，et al. Semi-supervised fault diagnosis method via graph label propagation and discriminative feature enhancement for critical components of industrial robot[J]. Journal of Mechanical Engineering，2022，58(17)：116-124.
[4] 刘赫，赵天成，刘俊博，等. 基于深度残差UNet网络的电气设备红外图像分割方法[J]. 红外技术，2022，44(12)：1351-1357. LIU He，ZHAO Tiancheng，LIU Junbo，et al. Deep residual unet network-based infrared lmage segmentation method for electrical equipment[J]. Infrared Technology，2022，44(12)：1351-1357.
[5] 王媛彬，李媛媛，段誉，等. 基于轻量骨干网络和注意力结构的变电设备红外图像识别[J]. 电网技术，2023，47(10)：4358-4369. WANG Yuanbin，LI Yuanyuan，DUAN Yu，et al. Infrared image recognition of substation equipment based on lightweight backbone network and attention mechanism[J]. Power System Technology，2023，47(10)：4358-4369.
[6] 梁剑，黄志鸿，张可人. 基于多尺度引导滤波和决策融合的电力设备热故障诊断方法研究[J]. 红外技术，2022，44(12)：1344-1350. LIANG Jian，HUANG Zhihong，ZHANG Keren. Multi-scale guided filter and decision fusion for thermal fault diagnosis of power equipment[J]. Infrared Technology，2022，44(12)：1344-1350.
[7] 高金吉. 工业互联网赋能装备智能运维与自主健康[J].计算机集成制造系统，2019，25(12)：3013-3025. GAO Jinji. Intelligent maintenance and autonomous health of equipments enabled by industrial internet[J]. Computer Integrated Manufacturing Systems，2019，25(12)：3013-3025.
[8] 易永率，赵海涛. 基于属性描述的多单元过程零样本故障诊断[J]. 华东理工大学学报，2023，49(6)：845-853. YI Yongshuai，ZAHO Haitao. Zero-sample fault diagnosis of multi-unit processes based on attribute description[J]. Journal of East China University of Science and Technology，2023，49(6)：845-853.
[9] 郭小萍，王浩，李元. 基于深度聚类的化工过程故障诊断[J]. 自动化与仪表，2023，38(1)：99-104. GUO Xiaoping，WANG Hao，LI Yuan. Fault diagnosis of chemical process based on deep clustering[J]. Automation & Instrumentation，2023，38(1)：99-104.
[10] 石静雯，侯立群. 基于一维卷积注意力门控循环网络和迁移学习的轴承故障诊断[J]. 振动与冲击，2023，42(3)：159-164. SHI Jingwen，HOU Liqun. Bearing fault diagnosis based on 1D CNN attention gated recurrent network and transfer learning[J]. Journal of Vibration and Shock，2023，42(3)：159-164.
[11] 瞿红春，朱伟华，高鹏宇，等. 基于注意力循环胶囊网络的滚动轴承故障诊断[J]. 振动.测试与诊断，2022，42(6)：1108-1114. QU Hongchun，ZHU Weihua，GAO Pengyu，et al. Fault diagnosis of rolling bearing based on attention recurrent capsule network[J]. Journal of Vibration，Measurement & Diagnosis，2022，42(6)：1108-1114.
[12] 吴铮，张悦，董泽. 基于多图卷积神经网络的主汽温系统故障诊断[J]. 动力工程学报，2023，43(2)：237-245. WU Zheng，ZHANG Rui，DONG Ze. Fault diagnosis of main steam temperature system based on multi-graph convolutional neural network[J]. Journal of Chinese Society of Power Engineering，2023，43(2)：237-245.
[13] 林昙涛，牛青波，马天旭，等. 基于Transformer的智能轴承声-振融合故障诊断[J]. 轴承，2023(2)：67-73. LIN Tantao，NIU Qingbo，MA Tianxu，et al. Acoustic-vibration fusion fault diagnosis for intelligent bearing based on transformer[J]. Bearing，2023(2)：67-73.
[14] 陈雪峰，王诗彬，程礼. 航空发动机快变信号的匹配同步压缩变换研究[J]. 机械工程学报，2019，55(13)：13-22. CHEN Xuefeng，WANG Shibin，CHENG Li. Matching synchrosqueezing transform for aero-engine’s signals with fast varying instantaneous frequency[J]. Journal of Mechanical Engineering，2019，55(13)：13-22.
[15] WANG F，ZHAO Z，ZHAI Z，et al. Explainability-driven model improvement for SOH estimation of lithium-ion battery[J]. Reliability Engineering & System Safety，2023，232：109046.
[16] 孙建波，王丽杰，麻吉辉，等. 基于改进YOLO v5s算法的光伏组件故障检测[J]. 红外技术，2023(2)：202-208. SUN Jianbo，WANG Lijie，MA Jihui，et al. Photovoltaic module fault detection based on lmproved YOLOv5s algorithm[J]. Infrared Technology, 2023(2)：202-208.
[17] WU J，ZHAO Z，SUN C，et al. Fault-attention generative probabilistic adversarial autoencoder for machine anomaly detection[J]. IEEE Transactions on Industrial Informatics，2020，16(12)：7479-7488.
[18] ZHOU Z，LI T，ZHANG Z，et al. Bayesian differentiable architecture search for efficient domain matching fault diagnosis[J]. IEEE Transactions on Instrumentation and Measurement，2021，70：1-11.
[19] HU C，WU J，SUN C，et al. Inter-instance and intra-temporal self-supervised learning with few labeled data for fault diagnosis[J]. IEEE Transactions on Industrial Informatics，2022.
[20] LI T，ZHOU Z，LI S，et al. The emerging graph neural networks for intelligent fault diagnostics and prognostics：A guideline and a benchmark study[J]. Mechanical Systems and Signal Processing，2022，168：108653.
[21] KHATAB A，DIALLO C，AGHEZZAF E H，et al. Condition-based selective maintenance for stochastically degrading multi-component systems under periodic inspection and imperfect maintenance[J]. Proceedings of the Institution of Mechanical Engineers，Part O：Journal of Risk and Reliability，2018，232(4)：447-463.
[22] XU W，ZHOU Z，LI T，et al. Physics-constraint variational neural network for wear state assessment of external gear pump[J]. IEEE Transactions on Neural Networks and Learning Systems，2024，35(5):5996-6006.
[23] ZHOU Z，LI T，ZHAO Z，et al. Time-varying trajectory modeling via dynamic governing network for remaining useful life prediction[J]. Mechanical Systems and Signal Processing，2023，182：109610.
[24] SHANG Z，ZHAO Z，YAN R. Denoising fault-aware wavelet network：A signal processing informed neural network for fault diagnosis[J]. Chinese Journal of Mechanical Engineering，2023，36：1-18.
[25] LI T，ZHAO Z，SUN C，et al. Wavelet kernel net：An interpretable deep neural network for industrial intelligent diagnosis[J]. IEEE Transactions on Systems，Man，and Cybernetics：Systems，2021，52(4)：2302-2312.
[26] ZHAO Z，LI T，AN B，et al. Model-driven deep unrolling：Towards interpretable deep learning against noise attacks for intelligent fault diagnosis[J]. ISA Transactions，2022，129：644-662.
[27] AN B，WANG S，ZHAO Z，et al. Interpretable neural network via algorithm unrolling for mechanical fault diagnosis[J]. IEEE Transactions on Instrumentation and Measurement，2022，71：1-11.
[28] LI X，ZHANG W，DING Q. Understanding and improving deep learning-based rolling bearing fault diagnosis with attention mechanism[J]. Signal processing，2019，161：136-154.
[29] JIANG J，LI H，MAO Z，et al. A digital twin auxiliary approach based on adaptive sparse attention network for diesel engine fault diagnosis[J]. Scientific Reports，2022，12(1)：675.
[30] MU K，LUO L，WANG Q，et al. Industrial process monitoring and fault diagnosis based on temporal attention augmented deep network[J]. Journal of Information Processing Systems，2021，17(2)：242-252.
[31] YANG Z，ZHANG J，ZHAO Z，et al. Interpreting network knowledge with attention mechanism for bearing fault diagnosis[J]. Applied Soft Computing，2020，97：106829.
[32] SHI Y，DENG A，DENG M，et al. Enhanced lightweight multiscale convolutional neural network for rolling bearing fault diagnosis[J]. IEEE Access，2020，8：217723-217734.
[33] LIU S，HUANG J，MA J，et al. SRMANet：Toward an interpretable neural network with multi-attention mechanism for gearbox fault diagnosis[J]. Applied Sciences，2022，12(16)：8388.
[34] WANG H，LIU Z，PENG D，et al. Understanding and learning discriminant features based on multiattention 1DCNN for wheelset bearing fault diagnosis[J]. IEEE Transactions on Industrial Informatics，2019，16(9)：5735-5745.
[35] FAN Z，XU X，WANG R，et al. Fan fault diagnosis based on lightweight multiscale multiattention feature fusion network[J]. IEEE Transactions on Industrial Informatics，2021，18(7)：4542-4554.
[36] VASWANI A，SHAZEER N，PARMAR N，et al. Attention is all you need[J]. Advances in Neural Information Processing Systems，2017，30：6000–6010.
[37] 房佳姝，刘崇茹，苏晨博，等. 基于自注意力Transformer编码器的多阶段电力系统暂态稳定评估方法[J]. 中国电机工程学报，2023，43(15)：5745-5759. FANG Jiashu，LIU Chongru，SU Chenbo，et al. Multi-stage transient stability assessment of power system based on self-attention transformer encoder[J]. Proceedings of the CSEE，2023，43(15)：5745-5759.
[38] BI X，ZHAO J. A novel orthogonal self-attentive variational autoencoder method for interpretable chemical process fault detection and identification[J]. Process Safety and Environmental Protection，2021，156：581-597.
[39] TANG J，ZHENG G，WEI C，et al. Signal-transformer：A robust and interpretable method for rotating machinery intelligent fault diagnosis under variable operating conditions[J]. IEEE Transactions on Instrumentation and Measurement，2022，71：1-11.
[40] LI Y，ZHOU Z，SUN C，et al. Variational attention-based interpretable transformer network for rotary machine fault diagnosis[J]. IEEE Transactions on Neural Networks and Learning Systems，2024，35(5)：6180-6193.
[41] LI S，LUO J，HU Y. Toward interpretable process monitoring：Slow feature analysis-aided autoencoder for spatiotemporal process feature learning[J]. IEEE Transactions on Instrumentation and Measurement，2021，71：1-11.
[42] LIAO J，DONG H，SUN Z，et al. Attention-embedded quadratic network (qttention) for effective and interpretable bearing fault diagnosis[J]. arXiv Preprint arXiv：2206.00390，2022.
[43] HAN L，DENG Y，CHEN H，et al. A robust VRF fault diagnosis method based on ensemble BiLSTM with attention mechanism：Considering uncertainties and generalization[J]. Energy and Buildings，2022，269：112243.
[44] GUO H，ZHANG Y，ZHU K. Interpretable deep learning approach for tool wear monitoring in high-speed milling[J]. Computers in Industry，2022，138：103638.
[45] 苏向敬，山衍浩，周汶鑫，等. 基于GRU和注意力机制的海上风机齿轮箱状态监测[J].电力系统保护与控制，2021，49(24)：141-149. SU Xiangjing，SHAN Yanhao，ZHOU Wenxin，et al. GRU and attention mechanism-based condition monitoring of an offshore wind turbine gearbox[J]. Power System Protection and Control，2021，49(24)：141-149.
[46] BRUSA E，DELPRETE C，DI MAGGIO L G. Eigen-spectrograms：An interpretable feature space for bearing fault diagnosis based on artificial intelligence and image processing[J]. Mechanics of Advanced Materials and Structures，2023，30(21-24)：4639-4651.
[47] ARELLANO-ESPITIA F，DELGADO-PRIETO M，MARTINEZ-VIOL V，et al. Diagnosis electromechanical system by means CNN and SAE：An interpretable-learning study[C]//2022 IEEE 5th International Conference on Industrial Cyber-Physical Systems (ICPS). IEEE，2022：1-6.
[48] MIRZAEI S，KANG J L，CHU K Y. A comparative study on long short-term memory and gated recurrent unit neural networks in fault diagnosis for chemical processes using visualization[J]. Journal of the Taiwan Institute of Chemical Engineers，2022，130：104028.
[49] COSTA N，SANCHEZ L. Variational encoding approach for interpretable assessment of remaining useful life estimation[J]. Reliability Engineering & System Safety，2022，222：108353.
[50] ZHANG D，CHEN Y，GUO F，et al. A new interpretable learning method for fault diagnosis of rolling bearings[J]. IEEE Transactions on Instrumentation and Measurement，2021，70：1-10.
[51] YANG X，ZHENG Y，ZHANG Y，et al. Bearing remaining useful life prediction based on regression shapalet and graph neural network[J]. IEEE Transactions on Instrumentation and Measurement，2022，71：1-12.
[52] ZHANG J，SONG X，GAO L，et al. L2-Norm shapelet dictionary learning-based bearing-fault diagnosis in uncertain working conditions[J]. IEEE Sensors Journal，2021，22(3)：2647-2657.
[53] JAIN S，WALLACE C B. Attention is not explanation[C]// Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics：Human Language Technologies，2019. Stroudsburg，PA：Association for Computational Linguistics，2019：3543-3556.
[54] WIEGREFFE S，PINTER Y. Attention is not not explanation[C]// Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)，2019. Stroudsburg，PA：Association for Computational Linguistics，2019：11-20.
[55] SERRANO S，SMITH N A. Is attention interpretable?[C]// Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics，2019. Stroudsburg，PA：Association for Computational Linguistics，2019：2931-2951.
[56] SUN K H，HUH H，TAMA B A，et al. Vision-based fault diagnostics using explainable deep learning with class activation maps[J]. IEEE Access，2020，8：129169-129179.
[57] LIU J，HOU L，WANG X，et al. Explainable fault diagnosis of gas-liquid separator based on fully convolutional neural network[J]. Computers & Chemical Engineering，2021，155：107535.
[58] CHEN Z，XU J，PENG T，et al. GCN-CAM：A new graph convolutional network-based fault diagnosis method with its interpretability analysis[C]//2021 CAA Symposium on Fault Detection，Supervision，and Safety for Technical Processes (SAFEPROCESS). IEEE，2021：1-6.
[59] YU S，WANG M，PANG S，et al. Intelligent fault diagnosis and visual interpretability of rotating machinery based on residual neural network[J]. Measurement，2022，196：111228.
[60] CHEN H Y，LEE C H. Vibration signals analysis by explainable artificial intelligence (XAI) approach：Application on bearing faults diagnosis[J]. IEEE Access，2020，8：134246-134256.
[61] BRITO L C，BRITO G A S J N，DUARTE M A V. Fault diagnosis using eXplainable AI：A transfer learning-based approach for rotating machinery exploiting augmented synthetic data[J]. arXiv Preprint arXiv：2210.02974，2022.
[62] CHEN B，LIU T，HE C，et al. Fault diagnosis for limited annotation signals and strong noise based on interpretable attention mechanism[J]. IEEE Sensors Journal，2022，22(12)：11865-11880.
[63] SUN H，CAO X，WANG C，et al. An interpretable anti-noise network for rolling bearing fault diagnosis based on FSWT[J]. Measurement，2022，190：110698.
[64] LI G，YAO Q，FAN C，et al. An explainable one-dimensional convolutional neural networks based fault diagnosis method for building heating，ventilation and air conditioning systems[J]. Building and Environment，2021，203：108057.
[65] YANG H，LI X，ZHANG W. Interpretability of deep convolutional neural networks on rolling bearing fault diagnosis[J]. Measurement Science and Technology，2022，33(5)：055005.
[66] LEE J，NOH I，LEE J，et al. Development of an explainable fault diagnosis framework based on sensor data imagification：A case study of the robotic spot-welding process[J]. IEEE Transactions on Industrial Informatics，2021，18(10)：6895-6904.
[67] 王冉，石如玉，胡升涵，等. 基于声成像与卷积神经网络的轴承故障诊断方法及其可解释性研究[J]. 振动与冲击，2022，41(16)：224-231. WANG Ran，SHI Ruyu，HU Shenghan，et al. An acoustic fault diagnosis method of rolling bearings based on acoustic imaging and convolutional neural network[J]. Journal of Vibration and Shock，2022，41(16)：224-231.
[68] BINDER A，MONTAVON G，LAPUSCHKIN S，et al. Layer-wise relevance propagation for neural networks with local renormalization layers[C]//Artificial Neural Networks and Machine Learning–ICANN 2016：25th International Conference on Artificial Neural Networks，Barcelona，Spain，September 6-9，2016，Proceedings，Part II 25. Springer International Publishing，2016：63-71.
[69] GREZMAK J，WANG P，SUN C，et al. Explainable convolutional neural network for gearbox fault diagnosis[J]. Procedia CIRP，2019，80：476-481.
[70] GREZMAK J，ZHANG J，WANG P，et al. Interpretable convolutional neural network through layer-wise relevance propagation for machine fault diagnosis[J]. IEEE Sensors Journal，2019，20(6)：3172-3181.
[71] GREZMAK J，ZHANG J，WANG P，et al. Multi-stream convolutional neural network-based fault diagnosis for variable frequency drives in sustainable manufacturing systems[J]. Procedia Manufacturing，2020，43：511-518.
[72] HAN J H，PARK S U，HONG S K. A study on the effectiveness of current data in motor mechanical fault diagnosis using XAI[J]. Journal of Electrical Engineering & Technology，2022，17(6)：3329-3335.
[73] WU H，HUANG A，SUTHERLAND J W. Layer-wise relevance propagation for interpreting LSTM-RNN decisions in predictive maintenance[J]. The International Journal of Advanced Manufacturing Technology，2022，118：963-978.
[74] SHI S. Bearing fault diagnosis based on interpretable sparse method[J]. International Journal of Science，2021，4(8)：267-276.
[75] LI G，LI F，XU C，et al. A spatial-temporal layer-wise relevance propagation method for improving interpretability and prediction accuracy of LSTM building energy prediction[J]. Energy and Buildings，2022，271：112317.
[76] AGARWAL P，TAMER M，BUDMAN H. Explainability：Relevance based dynamic deep learning algorithm for fault detection and diagnosis in chemical processes[J]. Computers & Chemical Engineering，2021，154：107467.
[77] QU H，WANG Y，ZHANG K. Bearing fault diagnosis based on ensemble depth explainable encoder classification model with arithmetic optimized tuning[J]. SSRN，2022，17：3979088.
[78] KIM I，KIM S W，KIM J，et al. Single domain generalizable and physically interpretable bearing fault diagnosis for unseen working conditions[J]. Expert Systems with Applications，2024，241：122455.
[79] WANG S，ZHANG Y. Multi-level federated network based on interpretable indicators for ship rolling bearing fault diagnosis[J]. Journal of Marine Science and Engineering，2022，10(6)：743.
[80] WONG S Y，YAP K S，YAP H J，et al. On equivalence of FIS and ELM for interpretable rule-based knowledge representation[J]. IEEE Transactions on Neural Networks and Learning Systems，2014，26(7)：1417-1430.
[81] YU J，LIU G. Knowledge extraction and insertion to deep belief network for gearbox fault diagnosis[J]. Knowledge-Based Systems，2020，197：105883.
[82] WU Z，LUO H，YANG Y，et al. K-PdM：KPI-oriented machinery deterioration estimation framework for predictive maintenance using cluster-based hidden Markov model[J]. IEEE Access，2018，6：41676-41687.
[83] STEENWINCKEL B，DE PAEPE D，HAUTTE S V，et al. FLAGS：A methodology for adaptive anomaly detection and root cause analysis on sensor data streams by fusing expert knowledge with machine learning[J]. Future Generation Computer Systems，2021，116：30-48.
[84] 陈果，李成刚，王德友. 利用神经网络规则提取方法获取转静碰摩故障诊断知识[J]. 航空学报，2008，29(5)：1319-1325. CHEN Guo，LI Chenggang，WANG Deyou. Rotor-stator rubbing gault fiagnosis knowledge acquisition using rule extraction from neural networks[J]. Acta Aeronautica ET Astronautica Sinica，2008，29(5)：1319-1325.
[85] 陈果. 神经网络规则提取及其在转子故障诊断中的应用研究[J]. 振动与冲击，2009，28(3)：59-62. CHEN Guo. Rule extraction form a neural network and its application in rotor fault diagnosis[J]. Journal of Vibration and Shock，2009，28(3)：59-62.
[86] 陈果，宋兰琪，陈立波. 基于神经网络规则提取的航空发动机磨损故障诊断知识获取[J]. 航空动力学报，2008，23(12)：2170-2176. CHEN Guo，SONG Lanqi，CHEN Libo. Knowledge acquisition for aero-engine wear fault diagnosis based on rule extraction from neural networks[J]. Journal of Aerospace Power，2008，23(12)：2170-2176.
[87] BHALLA D，BANSAL R K，GUPTA H O. Function analysis based rule extraction from artificial neural networks for transformer incipient fault diagnosis[J]. International Journal of Electrical Power & Energy Systems，2012，43(1)：1196-1203.
[88] BAPTISTA M，MISHRA M，HENRIQUES E，et al. Using explainable artificial intelligence to interpret remaining useful life estimation with gated recurrent unit[EB/OL]. [2023-03-29]. http://doi.10.13140/RG.2.2.27721.36963.
[89] SERRADILLA O，ZUGASTI E，CERNUDA C，et al. Interpreting remaining useful life estimations combining explainable artificial intelligence and domain knowledge in industrial machinery[C]//2020 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE). IEEE，2020：1-8.
[90] SANAKKAYALA D C，VARADARAJAN V，KUMAR N，et al. Explainable AI for bearing fault prognosis using deep learning techniques[J]. Micromachines，2022，13(19)：1471.
[91] PROTOPAPADAKIS G，APOSTOLIDIS A，KALFAS A I. Explainable and interpretable AI-assisted remaining useful life estimation for aeroengines[C/CD]//Turbo Expo：Power for Land，Sea，and Air. American Society of Mechanical Engineers，2022，Rotterdam，Netherlands，ASME，2022，85987-V002T05A002.
[92] GUPTA S，VENUGOPAL A，MOHAN M J. Fault detection and diagnosis using autoencoders and interpretable AI-case study on an industrial chiller[C]//2022 IEEE International Symposium on Advanced Control of Industrial Processes (AdCONIP). 2022，Vancouver，BC，Canada，IEEE，2022：198-203.
[93] MARTAKIS P，MOVSESSIAN A，REULAND Y，et al. A semi-supervised interpretable machine learning framework for sensor fault detection[J]. Smart Structures and Systems，2022，29：251-266.
[94] BAPTISTA M L，GOEBEL K，HENRIQUES E M P.. Relation between prognostics predictor evaluation metrics and local interpretability SHAP values[J]. Artificial Intelligence，2022，306：103667.
[95] NOR A K M. Failure prognostic of turbofan engines with uncertainty quantification and Explainable AI (XIA)[J]. Turkish Journal of Computer and Mathematics Education (TURCOMAT)，2021，12(3)：3494-3504.
[96] NOR A K M，PEDAPATI S R，MUHAMMAD M，et al. Explainable artificial intelligence for anomaly detection and prognostic of gas turbines using uncertainty quantification with sensor-related data explainable artificial intelligence for anomaly detection and prognostic of gas turbines using uncertainty Q[EB/OL]. [2023-3-29]. http://doi.10.20944/preprints202109.0034.v2.
[97] MOVSESSIAN A，CAVA D G，TCHERNIAK D. Interpretable machine learning in damage detection using shapley additive explanations[J]. ASCE-ASME Journal of Risk and Uncertainty in Engineering Systems，Part B：Mechanical Engineering，2022，8(2)：021101.
[98] JALALI A，SCHINDLER A，HASLHOFER B，et al. Machine learning interpretability techniques for outage prediction：A comparative study[C]//PHM Society European Conference. 2020，Virtual Conference，PHM Society，2020，5(1)：10.
[99] ONCHIS D M，GILLICH G R. Stable and explainable deep learning damage prediction for prismatic cantilever steel beam[J]. Computers in Industry，2021，125：103359.
[100] Case Western Reserve University，Bearing data center[EB/OL]. [2023-3-29]. https：//engineering.case. edu/bearingdatacenter.
[101] 张俊鹏，杨志勃，陈雪峰，等. 卷积神经网络在轴承故障诊断中的可解释性探讨[J]. 轴承，2020(7)：54-60. ZHANG Junpeng，YANG Zhibo，CHEN Xuefeng，et al. Interpretability discussion on convolutional neural network in bearing fault diagnosis[J]. Bearing，2020(7)：54-60.
[102] HERWIG N，BORGHESANI P. Explaining deep neural networks processing raw diagnostic signals[J]. Mechanical Systems and Signal Processing，2023，200：110584.
[103] MIKUT R，JÄKEL J，GRÖLL L. Interpretability issues in data-based learning of fuzzy systems[J]. Fuzzy Sets and Systems，2005，150(2)：179-197.
[104] MILLER G A. The magical number seven，plus or minus two：Some limits on our capacity for processing information[J]. Psychological Review，1956，63(2)：81-97.
[105] BAU D，ZHOU B，KHOSLA A，et al. Network dissection：Quantifying interpretability of deep visual representations[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017：6541-6549.