Reinforcement Learning-based Swarm Evolutionary Algorithm To Solve Two-sided Multi-objective Synchronous Parallel Disassembly Line Balancing Problem

doi:10.3901/JME.2023.07.355

Abstract

Abstract: Under the background of dual carbon, the industrial manufacturing field is transforming in the direction of green and energy-saving. The recycling, disassembling and remanufacturing of waste products is conducive to promoting high-quality development. Aiming at the problem that it is difficult to change the disassembly direction during the disassembly process of large and complex products, and comprehensively considering the various constraints that exist between tasks in the actual disassembly process, the paper studies the two-sided disassembly line balancing problem (TDLBP) in the synchronous parallel mode. Firstly, the disassembly line mode of two-sided layout is introduced, and the relationship of and priority and or priority is defined, and the mathematical model of TDLBP is established to optimize the layout of the production line, economic benefits, safety and environmental protection with a total of six indicators. Then, a swarm evolutionary algorithm based on reinforcement learning is proposed. Q-learning is used to select the best operator in the iteration by using the knowledge learned, the Pareto solution set is screened by the crowding distance, and the elite retention strategy is used to accelerate the convergence of the algorithm, so as to efficiently obtain the approximate optimal disassembly scheme. Finally, the effectiveness and superiority of the proposed algorithm are verified by solving small-scale cases and comparative analysis, and the application of large-scale case is carried out.

Key words: two-sided disassembly line, synchronous parallel, reinforcement learning, swarm evolutionary algorithm, Pareto

CLC Number:

TH165

GUO Hongfei, LU Xinyu, REN Yaping, ZHANG Chaoyong, LI Jianqing. Reinforcement Learning-based Swarm Evolutionary Algorithm To Solve Two-sided Multi-objective Synchronous Parallel Disassembly Line Balancing Problem[J]. Journal of Mechanical Engineering, 2023, 59(7): 355-366.

References

[1] GUO H,ZHANG L,ZHANG Y,et al. A differential game-based approach for school-enterprise collaborative R&D strategy on digital twin technology[J]. IEEE Access,2020,8:188688-188698.
[2] REN Y,JIN H,ZHAO F,et al. A multiobjective disassembly planning for value recovery and energy conservation from end-of-life products[J]. IEEE Transactions on Automation Science and Engineering,2020,18(2):791-803.
[3] MA J,CHEN H,ZHANG Y,et al. A digital twin-driven production management system for production workshop[J]. The International Journal of Advanced Manufacturing Technology,2020,110(5):1385-1397.
[4] 郭洪飞,陈志彬,任亚平,等.基于零件回收综合评价的废旧产品拆解序列与拆解深度集成决策研究[J].机械工程学报,2022,58(4):258-268.GUO Hongfei,CHEN Zhibin,REN Yaping,et al. Research on disassembly sequence and disassembly length integrated decision of end-of-life products based on parts recovery comprehensive evaluation[J]. Journal of Mechanical Engineering,2022,58(4):258-268.
[5] 任亚平,郭洪飞,张超勇,等. 考虑产品制造过程内含能的选择性拆解规划能耗优化研究[J]. 机械工程学报,2021,57(6):200-210.REN Yaping,GUO Hongfei,ZHANG Chaoyong,et al. Energy consumption optimization of selective disassembly planning considering product embodied energy during manufacturing[J]. Journal of Mechanical Engineering,2021,57(6):200-210.
[6] REN Y,ZHANG C,ZHAO F,et al. An MCDM-based multiobjective general variable neighborhood search approach for disassembly line balancing problem[J]. IEEE Transactions on Systems,Man,and Cybernetics:Systems,2018,50(10):3770-3783.
[7] GUO H,LIAN X,ZHANG Y,et al. Analysis of environmental policy's impact on remanufacturing decision under the effect of green network using differential game model[J]. IEEE Access,2020,8:115251-115262.
[8] ÖZCEYLAN E,KALAYCI C B,GÜNGÖR A,et al. Disassembly line balancing problem:a review of the state of the art and future directions[J]. International Journal of Production Research,2019,57(15-16):4805-4827.
[9] 任亚平. 废旧产品拆解序列规划问题建模与优化研究[D]. 武汉:华中科技大学,2019.REN Yaping. Research on the modeling and optimization of disassembly sequence planning problem for used products[D]. Wuhan:Huazhong University of Science and Technology,2019.
[10] GÜNGÖR A,GUPTA S M. Disassembly line balancing[C]//Proceedings of the Annual Meeting of the Northeast Decision Sciences Institute,1999,Newport,RI.
[11] GUPTA S M,GUNGOR A. Product recovery using a disassembly line:Challenges and solution[C]//Proceedings of the 2001 IEEE International Symposium on Electronics and the Environment. 2001 IEEE ISEE (Cat. No. 01CH37190). IEEE,2001:36-40.
[12] AGRAWAL S,TIWARI M K. A collaborative ant colony algorithm to stochastic mixed-model U-shaped disassembly line balancing and sequencing problem[J]. International Journal of Production Research,2008,46(6):1405-1429.
[13] WANG K,LI X,GAO L. A multi-objective discrete flower pollination algorithm for stochastic two-sided partial disassembly line balancing problem[J]. Computers & Industrial Engineering,2019,130:634-649.
[14] ZHANG Y,ZHANG Z,GUAN C,et al. Improved whale optimization algorithm for two-sided disassembly line balancing problems considering part characteristic indexes[J]. International Journal of Production Research,2021:1-19.
[15] MUTLU S,GÜNER B. A memetic algorithm for mixed-model two-sided disassembly line balancing problem[J]. Procedia CIRP,2021,98:67-72.
[16] LIANG J,GUO S,DU B,et al. Minimizing energy consumption in multi-objective two-sided disassembly line balancing problem with complex execution constraints using dual-individual simulated annealing algorithm[J]. Journal of Cleaner Production,2021,284:125418.
[17] 尹凤福,杜泽瑞,李林,等. 基于双种群遗传算法的废旧智能手机拆卸序列规划[J]. 机械工程学报,2021,57(17):226-235. YIN Fengfu,DU Zerui,LI Lin,et al. Disassembly sequence planning of used smartphone based on dual-population genetic algorithm[J]. Journal of Mechanical Engineering,2021,57(17):226-235.
[18] 邢宇飞,王成恩,柳强. 基于Pareto解集蚁群算法的拆卸序列规划[J]. 机械工程学报,2012,48(9):186-192.XING Yufei,WANG Chengen,LIU Qiang. Disassembly sequence planning based on pareto ant colony algorithm[J]. Journal of Mechanical Engineering,2012,48(9):186-192.
[19] REN Y,YU D,ZHANG C,et al. An improved gravitational search algorithm for profit-oriented partial disassembly line balancing problem[J]. International Journal of Production Research,2017,55(24):7302-7316.
[20] REN Y,ZHANG C,ZHAO F,et al. Disassembly line balancing problem using interdependent weights-based multi-criteria decision making and 2-Optimal algorithm[J]. Journal of Cleaner Production,2018,174:1475-1486.
[21] ZHANG Z,WANG K,ZHU L,et al. A Pareto improved artificial fish swarm algorithm for solving a multi-objective fuzzy disassembly line balancing problem[J]. Expert Systems with Applications,2017,86:165-176.
[22] KALAYCI C B,GUPTA S M,NAKASHIMA K. Bees colony intelligence in solving disassembly line balancing problem[C]//Proceedings of the 2011 Asian Conference of Management Science and Applications,Sanya,China. 2011:21-23.
[23] MORIMOTO J,DOYA K. Robust reinforcement learning[J]. Neural computation,2005,17(2):335-359.
[24] TEH Y,BAPST V,CZARNECKI W M,et al. Distral:Robust multitask reinforcement learning[J]. Advances in neural information processing systems,2017,30.
[25] KAELBLING L P,LITTMAN M L,MOORE A W. Reinforcement learning:A survey[J]. Journal of artificial intelligence research,1996,4:237-285.
[26] TUNCEL E,ZEID A,KAMARTHI S. Solving large scale disassembly line balancing problem with uncertainty using reinforcement learning[J]. Journal of Intelligent Manufacturing,2014,25(4):647-659.
[27] BIERWIRTH C,MATTFELD D C,KOPFER H. On permutation representations for scheduling problems[C]//International Conference on Parallel Problem Solving from Nature. Springer,Berlin,Heidelberg,1996:310-318.
[28] DEB K,PRATAP A,AGARWAL S,et al. A fast and elitist multiobjective genetic algorithm:NSGA-II[J]. IEEE transactions on evolutionary computation,2002,6(2):182-197.
[29] 邹宾森,张则强,李六柯,等. 双边拆解线平衡问题建模与优化[J]. 中国机械工程,2018,29(9):1090-1097,1107.ZOU Binsen,ZHANG Zeqiang,LI Liuke,et al. Modeling and optimization for two-sided disassembly line balancing problems[J]. China Mechanical Engineering,2018,29(9):1090-1097,1107.
[30] 曾艳清,张则强,刘俊琦,等. 部分破坏性拆解线平衡问题建模与优化[J]. 信息与控制,2020,49(3):365-376.ZENG Yanqing, ZHANG Zeqiang, LIU Junqi,et al. Modeling and optimization of partial destructive disassembly-line balancing problem[J]. Information and Control,2020,49(3):365-376.