Domain Generalization D3QN for Machinery Fault Diagnosis Across Different Working Conditions

doi:10.3901/JME.2024.22.165

Abstract

Abstract: To address the problem of poor portability of deep reinforcement learning model in cross-condition fault diagnosis due to its dependence on the interaction environment, a domain generalization D3QN (Domain generalization dueling double deep Q network, DGD3QN)model is proposed for the machinery fault diagnosis across different working conditions. To realize the de-redundancy and refinement of data environment, the adaptive weighted max-relevance-min-redundancy method is utilized to optimize feature selection. The domain recognition network branch is introduced into D3QN network to separate and extract the fault state information from multi-conditions. To enhance the agent’s ability of identifying the overlapping failure modes in the multi-condition, the graded reward strategy is set by combining the domain recognition reward and the quantitative reward matrix constructed based on the inter-class distance of multi-condition failure modes. The experimental results of cross-condition diagnosis of gearbox fault and bearing fault showed that the proposed DGD3QN can better solve the contradiction between the environment dependence of DQN and the independence of cross-condition fault diagnosis on environmental conditions, realize the multiplexing and transplantation of D3QN models in different operating environments and enhance the applicability of DQN in the cross-domain fault diagnosis accuracy.

Key words: fault diagnosis, domain generalization, feature screening, graded reward strategy, deep reinforcement learning

CLC Number:

TH133

BO Lin, HE Mugeng, CHEN Bingkui, LIU Xiaofeng. Domain Generalization D3QN for Machinery Fault Diagnosis Across Different Working Conditions[J]. Journal of Mechanical Engineering, 2024, 60(22): 165-178.

References

[1] 宋秉睿. 基于机器学习的配电网故障定位及负荷转供研究[D]. 北京：北京交通大学，2021. SONG Bingrui. Research on fault location and load transfer in distribution network based on machine learning[D]. Beijing：Beijing Jiaotong University，2021.
[2] SHANTIA A，TIMMERS R，CHONG Y，et al. Two-stage visual navigation by deep neural networks and multi-goal reinforcement learning[J]. Robotics and Autonomous Systems，2021，138：103731.
[3] LAI Y H，WU T C，LAI C F，et al. Cognitive optimal-setting control of a IoT industrial applications with deep reinforcement learning[J]. IEEE Transactions on Industrial Informatics，2020，17(3)：2116-2123.
[4] IBRAHIM A M，YAU K L A，CHONG Y W，et al. Applications of multi-agent deep reinforcement learning: models and algorithms[J]. Applied Sciences，2021，11(22)：10870.
[5] ZHOU S K，LE H N，LUU K，et al. Deep reinforcement learning in medical imaging：A literature review[J]. Medical Image Analysis，2021，73：102193.
[6] LEE J，KOH H，CHOE H J. Learning to trade in financial time series using high-frequency through wavelet transformation and deep reinforcement learning[J]. Applied Intelligence，2021，51(8)：6202-6223.
[7] DAI W，MO Z，LUO C，et al. Fault diagnosis of rotating machinery based on deep reinforcement learning and reciprocal of smoothness index[J]. IEEE Sensors Journal，2020，20(15)：8307-8315.
[8] WEN L，LI X，GAO L. A new reinforcement learning based learning rate scheduler for convolutional neural network in fault classification[J]. IEEE Transactions on Industrial Electronics，2020，68(12)：12890-12900.
[9] ZHANG W，FENG W，ZHAO H，et al. Rapidly learning bayesian networks for complex system diagnosis：A reinforcement learning directed greedy search approach[J]. IEEE Access，2019，8：2813-2823.
[10] 辛阔，王建国，张文兴. 基于深度Q学习策略的旋转机械故障诊断方法研究[J]. 机电工程，2021，38(10)：1261-1268. XIN Kuo，WANG Jianguo，ZHANG Wenxing. Research on fault diagnosis method of rotating machinery based on deep Q-learning strategy[J]. Mechatronics Engineering，2021，38(10)：1261-1268.
[11] DING Y，MA L，MA J，et al. Intelligent fault diagnosis for rotating machinery using deep Q-network based health state classification：A deep reinforcement learning approach[J]. Advanced Engineering Informatics，2019，42：100977.
[12] 陈仁祥，周君，胡小林，等. 多Agent深度Q学习和模糊积分的行星齿轮箱故障诊断[J]. 振动与冲击，2021，40(11)：147-153. CHEN Renxiang，ZHOU Jun，HU Xiaolin，et al. Multi-agent deep Q-learning and fuzzy integration for planetary gearbox fault diagnosis[J]. Vibration and Shock，2021，40(11)：147-153.
[13] WU Z，JIANG H，LIU S，et al. A deep reinforcement transfer convolutional neural network for rolling bearing fault diagnosis[J]. ISA Transactions，2022，129：505-524.
[14] LI G，WU J，DENG C，et al. Deep reinforcement learning-based online Domain adaptation method for fault diagnosis of rotating machinery[J]. IEEE/ASME Transactions on Mechatronics，2022，27(5)：2796-2805.
[15] FAN S，ZHANG X，SONG Z. Imbalanced sample selection with deep reinforcement learning for fault diagnosis[J]. IEEE Transactions on Industrial Informatics，2021，18(4)：2518-2527.
[16] 康守强，刘哲，王玉静，等. 基于改进DQN网络的滚动轴承故障诊断方法[J]. 仪器仪表学报，2021，42(3)：201-212. KANG Shouqiang，LIU Zhe，WANG Yujing，et al. A rolling bearing fault diagnosis method based on improved DQN network[J]. Journal of Instrumentation，2021，42(3)：201-212.
[17] MNIH V，KAVUKCUOGLU K，SILVER D，et al. Human- level control through deep reinforcement learning[J]. Nature 2015，518(2015)：529-533.
[18] YU Dinga，LIANG Maa，JIAN Ma，et al. Intelligent fault diagnosis for rotating machinery using deep Q-network based health state classification：A deep reinforcement learning approach[J]. Advanced Engineering Informatics 2019，42(2019)：100977
[19] 袁泉，曾文驱，李子涵，等. 基于改进型D3QN深度强化学习的铁路智能选线方法[J]. 铁道科学与工程学报，2022，19(2)：344-350. YUAN Quan，ZENG Wenqu，LI Zihan，et al. Intelligent railroad routing method based on improved D3QN deep reinforceent learning[J]. Journal of Railway Science and Engineering，2022，19(2)：344-350.
[20] 李长松，刘凯，肖先勇，等. 基于条件互信息特征选择法和Adaboost算法的电能质量复合扰动分类[J]. 高电压技术，2019，45(2)：579-585. LI Changsong，LIU Kai，XIAO Xianyong，et al. Power quality composite disturbance classification based on conditional mutual information feature selection method and Adaboost algorithm[J]. High Voltage Technology，2019，45(2)：579-585.
[21] 谢彦红，孙呈敖，李元. 加权动态SVDD在非线性过程监测中的应用研究[J]. 系统仿真学报，2017，29(7)：1506-1513. XIE Yanhong，SUN Chengao，LI Yuan. Research on the application of weighted dynamic SVDD in nonlinear process monitoring[J]. Journal of System Simulation，2017，29(7)：1506-1513.
[22] TANG Z，BO L，LIU X，et al. A semisupervised transferable lstm with feature evaluation for fault diagnosis of rotating machinery[J]. Applied Intelligence，2022，52(2)：1703-1717.
[23] GANIN Y，USTINOVA E，AJAKAN H，et al. Domain adversarial training of neural networks[J]. The Journal of Machine Learning Research，2016，17(1)：2096-2130.