基于强化学习的群体进化算法求解双边多目标同步并行拆解线平衡问题

doi:10.3901/JME.2023.07.355

机械工程学报 ›› 2023, Vol. 59 ›› Issue (7): 355-366.doi: 10.3901/JME.2023.07.355

• 资源化与再制造技术及装备 • 上一篇下一篇

扫码分享

基于强化学习的群体进化算法求解双边多目标同步并行拆解线平衡问题

郭洪飞^1,2, 陆鑫宇³, 任亚平^1,2, 张超勇⁴, 李建庆⁵

1. 暨南大学智能科学与工程学院珠海 519070;
2. 暨南大学物联网与物流工程研究院珠海 519070;
3. 暨南大学管理学院广州 510632;
4. 华中科技大学数字制造装备与技术国家重点实验室武汉 430074;
5. 澳门科技大学计算机科学与工程学院澳门 999078

收稿日期:2022-05-23 修回日期:2022-10-11 出版日期:2023-04-05 发布日期:2023-06-16
通讯作者: 任亚平(通信作者),男,1995年出生,博士,副教授,硕士研究生导师。主要研究方向为可持续设计与制造、产品拆解决策理论与方法、优化算法设计与应用。E-mail:renyp1@163.com
作者简介:郭洪飞,男,1980年出生,博士,副教授,博士研究生导师。主要研究方向为工业工程、智能制造及军民融合等。E-mail:ghf-2005@163.com
基金资助:
国家自然科学基金(52205526，52205538)、科技部国家外国专家(G2021199026L)、广州市基础研究计划基础与应用基础研究(202201010284)、广东省研究生教育创新计划(82620516)、广州市创新领军团队(201909010006)和广东省“质量工程”建设(210308)资助项目

Reinforcement Learning-based Swarm Evolutionary Algorithm To Solve Two-sided Multi-objective Synchronous Parallel Disassembly Line Balancing Problem

GUO Hongfei^1,2, LU Xinyu³, REN Yaping^1,2, ZHANG Chaoyong⁴, LI Jianqing⁵

1. School of Intelligent Systems Science and Engineering, Jinan University, Zhuhai 519070;
2. Institute of Physical Internet, Jinan University, Zhuhai 519070;
3. School of Management, Jinan University, Guangzhou 510632;
4. State Key Lab of Digital Manufacturing Equipment and Technology, Huazhong University of Science and Technology, Wuhan 430074;
5. School of Computer Science and Engineering, Macau University of Science and Technology, Macau 999078

Received:2022-05-23 Revised:2022-10-11 Online:2023-04-05 Published:2023-06-16

摘要/Abstract

摘要： 双碳背景下，工业制造领域朝着绿色、节能的方向转型，对废旧产品进行回收拆解以及再制造有利于推动高质量发展。论文针对大型复杂产品拆解过程中难以改变拆卸方向的问题，并综合考虑实际拆卸过程中任务之间存在的多种约束类型，研究了同步并行模式下的双边拆解线平衡问题(two-sided disassembly line balancing problem, TDLBP)。首先引入双边布局的拆解线模式，定义了与优先和或优先关系，建立了TDLBP的数学模型以优化产线布置、经济效益和安全环保三个方面共六个指标。然后提出了一种基于强化学习的群体进化算法，采用Q-learing利用所学知识选择迭代中的最佳算子，通过拥挤距离筛选Pareto解集，利用精英保留策略加速算法收敛，进而高效获取近似最优的拆解方案。最后通过求解小规模算例并对比分析，验证了所提出算法的有效性和优越性，并进行大规模案例的应用。

关键词: 双边拆解线, 同步并行, 强化学习, 群体进化算法, Pareto

Abstract: Under the background of dual carbon, the industrial manufacturing field is transforming in the direction of green and energy-saving. The recycling, disassembling and remanufacturing of waste products is conducive to promoting high-quality development. Aiming at the problem that it is difficult to change the disassembly direction during the disassembly process of large and complex products, and comprehensively considering the various constraints that exist between tasks in the actual disassembly process, the paper studies the two-sided disassembly line balancing problem (TDLBP) in the synchronous parallel mode. Firstly, the disassembly line mode of two-sided layout is introduced, and the relationship of and priority and or priority is defined, and the mathematical model of TDLBP is established to optimize the layout of the production line, economic benefits, safety and environmental protection with a total of six indicators. Then, a swarm evolutionary algorithm based on reinforcement learning is proposed. Q-learning is used to select the best operator in the iteration by using the knowledge learned, the Pareto solution set is screened by the crowding distance, and the elite retention strategy is used to accelerate the convergence of the algorithm, so as to efficiently obtain the approximate optimal disassembly scheme. Finally, the effectiveness and superiority of the proposed algorithm are verified by solving small-scale cases and comparative analysis, and the application of large-scale case is carried out.

Key words: two-sided disassembly line, synchronous parallel, reinforcement learning, swarm evolutionary algorithm, Pareto

中图分类号:

TH165

郭洪飞, 陆鑫宇, 任亚平, 张超勇, 李建庆. 基于强化学习的群体进化算法求解双边多目标同步并行拆解线平衡问题[J]. 机械工程学报, 2023, 59(7): 355-366.

GUO Hongfei, LU Xinyu, REN Yaping, ZHANG Chaoyong, LI Jianqing. Reinforcement Learning-based Swarm Evolutionary Algorithm To Solve Two-sided Multi-objective Synchronous Parallel Disassembly Line Balancing Problem[J]. Journal of Mechanical Engineering, 2023, 59(7): 355-366.

参考文献

[1] GUO H,ZHANG L,ZHANG Y,et al. A differential game-based approach for school-enterprise collaborative R&D strategy on digital twin technology[J]. IEEE Access,2020,8:188688-188698.
[2] REN Y,JIN H,ZHAO F,et al. A multiobjective disassembly planning for value recovery and energy conservation from end-of-life products[J]. IEEE Transactions on Automation Science and Engineering,2020,18(2):791-803.
[3] MA J,CHEN H,ZHANG Y,et al. A digital twin-driven production management system for production workshop[J]. The International Journal of Advanced Manufacturing Technology,2020,110(5):1385-1397.
[4] 郭洪飞,陈志彬,任亚平,等.基于零件回收综合评价的废旧产品拆解序列与拆解深度集成决策研究[J].机械工程学报,2022,58(4):258-268.GUO Hongfei,CHEN Zhibin,REN Yaping,et al. Research on disassembly sequence and disassembly length integrated decision of end-of-life products based on parts recovery comprehensive evaluation[J]. Journal of Mechanical Engineering,2022,58(4):258-268.
[5] 任亚平,郭洪飞,张超勇,等. 考虑产品制造过程内含能的选择性拆解规划能耗优化研究[J]. 机械工程学报,2021,57(6):200-210.REN Yaping,GUO Hongfei,ZHANG Chaoyong,et al. Energy consumption optimization of selective disassembly planning considering product embodied energy during manufacturing[J]. Journal of Mechanical Engineering,2021,57(6):200-210.
[6] REN Y,ZHANG C,ZHAO F,et al. An MCDM-based multiobjective general variable neighborhood search approach for disassembly line balancing problem[J]. IEEE Transactions on Systems,Man,and Cybernetics:Systems,2018,50(10):3770-3783.
[7] GUO H,LIAN X,ZHANG Y,et al. Analysis of environmental policy's impact on remanufacturing decision under the effect of green network using differential game model[J]. IEEE Access,2020,8:115251-115262.
[8] ÖZCEYLAN E,KALAYCI C B,GÜNGÖR A,et al. Disassembly line balancing problem:a review of the state of the art and future directions[J]. International Journal of Production Research,2019,57(15-16):4805-4827.
[9] 任亚平. 废旧产品拆解序列规划问题建模与优化研究[D]. 武汉:华中科技大学,2019.REN Yaping. Research on the modeling and optimization of disassembly sequence planning problem for used products[D]. Wuhan:Huazhong University of Science and Technology,2019.
[10] GÜNGÖR A,GUPTA S M. Disassembly line balancing[C]//Proceedings of the Annual Meeting of the Northeast Decision Sciences Institute,1999,Newport,RI.
[11] GUPTA S M,GUNGOR A. Product recovery using a disassembly line:Challenges and solution[C]//Proceedings of the 2001 IEEE International Symposium on Electronics and the Environment. 2001 IEEE ISEE (Cat. No. 01CH37190). IEEE,2001:36-40.
[12] AGRAWAL S,TIWARI M K. A collaborative ant colony algorithm to stochastic mixed-model U-shaped disassembly line balancing and sequencing problem[J]. International Journal of Production Research,2008,46(6):1405-1429.
[13] WANG K,LI X,GAO L. A multi-objective discrete flower pollination algorithm for stochastic two-sided partial disassembly line balancing problem[J]. Computers & Industrial Engineering,2019,130:634-649.
[14] ZHANG Y,ZHANG Z,GUAN C,et al. Improved whale optimization algorithm for two-sided disassembly line balancing problems considering part characteristic indexes[J]. International Journal of Production Research,2021:1-19.
[15] MUTLU S,GÜNER B. A memetic algorithm for mixed-model two-sided disassembly line balancing problem[J]. Procedia CIRP,2021,98:67-72.
[16] LIANG J,GUO S,DU B,et al. Minimizing energy consumption in multi-objective two-sided disassembly line balancing problem with complex execution constraints using dual-individual simulated annealing algorithm[J]. Journal of Cleaner Production,2021,284:125418.
[17] 尹凤福,杜泽瑞,李林,等. 基于双种群遗传算法的废旧智能手机拆卸序列规划[J]. 机械工程学报,2021,57(17):226-235. YIN Fengfu,DU Zerui,LI Lin,et al. Disassembly sequence planning of used smartphone based on dual-population genetic algorithm[J]. Journal of Mechanical Engineering,2021,57(17):226-235.
[18] 邢宇飞,王成恩,柳强. 基于Pareto解集蚁群算法的拆卸序列规划[J]. 机械工程学报,2012,48(9):186-192.XING Yufei,WANG Chengen,LIU Qiang. Disassembly sequence planning based on pareto ant colony algorithm[J]. Journal of Mechanical Engineering,2012,48(9):186-192.
[19] REN Y,YU D,ZHANG C,et al. An improved gravitational search algorithm for profit-oriented partial disassembly line balancing problem[J]. International Journal of Production Research,2017,55(24):7302-7316.
[20] REN Y,ZHANG C,ZHAO F,et al. Disassembly line balancing problem using interdependent weights-based multi-criteria decision making and 2-Optimal algorithm[J]. Journal of Cleaner Production,2018,174:1475-1486.
[21] ZHANG Z,WANG K,ZHU L,et al. A Pareto improved artificial fish swarm algorithm for solving a multi-objective fuzzy disassembly line balancing problem[J]. Expert Systems with Applications,2017,86:165-176.
[22] KALAYCI C B,GUPTA S M,NAKASHIMA K. Bees colony intelligence in solving disassembly line balancing problem[C]//Proceedings of the 2011 Asian Conference of Management Science and Applications,Sanya,China. 2011:21-23.
[23] MORIMOTO J,DOYA K. Robust reinforcement learning[J]. Neural computation,2005,17(2):335-359.
[24] TEH Y,BAPST V,CZARNECKI W M,et al. Distral:Robust multitask reinforcement learning[J]. Advances in neural information processing systems,2017,30.
[25] KAELBLING L P,LITTMAN M L,MOORE A W. Reinforcement learning:A survey[J]. Journal of artificial intelligence research,1996,4:237-285.
[26] TUNCEL E,ZEID A,KAMARTHI S. Solving large scale disassembly line balancing problem with uncertainty using reinforcement learning[J]. Journal of Intelligent Manufacturing,2014,25(4):647-659.
[27] BIERWIRTH C,MATTFELD D C,KOPFER H. On permutation representations for scheduling problems[C]//International Conference on Parallel Problem Solving from Nature. Springer,Berlin,Heidelberg,1996:310-318.
[28] DEB K,PRATAP A,AGARWAL S,et al. A fast and elitist multiobjective genetic algorithm:NSGA-II[J]. IEEE transactions on evolutionary computation,2002,6(2):182-197.
[29] 邹宾森,张则强,李六柯,等. 双边拆解线平衡问题建模与优化[J]. 中国机械工程,2018,29(9):1090-1097,1107.ZOU Binsen,ZHANG Zeqiang,LI Liuke,et al. Modeling and optimization for two-sided disassembly line balancing problems[J]. China Mechanical Engineering,2018,29(9):1090-1097,1107.
[30] 曾艳清,张则强,刘俊琦,等. 部分破坏性拆解线平衡问题建模与优化[J]. 信息与控制,2020,49(3):365-376.ZENG Yanqing, ZHANG Zeqiang, LIU Junqi,et al. Modeling and optimization of partial destructive disassembly-line balancing problem[J]. Information and Control,2020,49(3):365-376.

基于强化学习的群体进化算法求解双边多目标同步并行拆解线平衡问题

Reinforcement Learning-based Swarm Evolutionary Algorithm To Solve Two-sided Multi-objective Synchronous Parallel Disassembly Line Balancing Problem

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

本文评价

[1]	王无印, 黄子钊, 庄子龙, 方怀瑾, 秦威. 基于深度强化学习的自动化码头堆场场桥调度方法[J]. 机械工程学报, 2024, 60(6): 44-57.
[2]	赵阔, 王皂琦, 潘臻信, 潘扬华, 张中飞, 屈挺. 大数据驱动的快消品终端拜访“云-边”联动决策与优化[J]. 机械工程学报, 2024, 60(6): 58-68.
[3]	郭景华, 李文昌, 王班, 王靖瑶. 基于深度强化学习的网联混合动力汽车队列控制[J]. 机械工程学报, 2024, 60(2): 262-271.
[4]	李文礼, 张祎楠, 石晓辉, 王梦昕. 基于博弈论的右转无信号交叉口行人行为模拟[J]. 机械工程学报, 2024, 60(10): 86-101.
[5]	曾迪, 郑玲, 李以农, 杨显通. 自动驾驶奖励函数贝叶斯逆强化学习方法[J]. 机械工程学报, 2024, 60(10): 245-260.
[6]	杨硕, 李时珍, 赵中原, 黄小鹏, 黄岩军. 基于时序差分学习模型预测控制的一体化自动驾驶换道策略[J]. 机械工程学报, 2024, 60(10): 329-338.
[7]	王娟, 刘美红, 祝世兴, 陈文博, 李遇贤, 孙军锋. 指尖密封结构参数的多目标混合教与学优化方法[J]. 机械工程学报, 2023, 59(9): 157-170.
[8]	张志勇, 黄大洋, 黄彩霞, 胡林, 杜荣华. TD3算法改进与自动驾驶汽车并道策略学习[J]. 机械工程学报, 2023, 59(8): 224-234.
[9]	郑湃, 李成熙, 殷悦, 张荣, 鲍劲松, 王柏村, 谢海波, 王力翚. 增强现实辅助的互认知人机安全交互系统[J]. 机械工程学报, 2023, 59(6): 173-184.
[10]	娄山河, 冯毅雄, 郑浩, 胡炳涛, 洪兆溪, 谭建荣. 模拟人脑工作机制的机械产品概念设计行为原理启发求解方法[J]. 机械工程学报, 2023, 59(24): 344-358.
[11]	马丽莹, 魏云冰. 基于DDPG算法的发电企业报价策略研究[J]. 电气工程学报, 2023, 18(2): 192-200.
[12]	林歆悠, 叶卓明, 周斌豪. 基于DQN强化学习的自动驾驶转向控制策略[J]. 机械工程学报, 2023, 59(16): 315-324.
[13]	钟沛成, 骆德渊, 庞明君. 基于深度强化学习的四足机器人跟随策略研究及系统实现[J]. 机械工程学报, 2023, 59(13): 79-88.
[14]	顾文斌, 李育鑫, 刘斯麒, 苑明海, 裴凤雀. 数据驱动的智慧车间实时调度方法研究[J]. 机械工程学报, 2023, 59(12): 47-61.
[15]	陈睿奇, 黎雯馨, 王传洋, 杨宏兵. 基于深度强化学习的工序交互式智能体Job shop调度方法[J]. 机械工程学报, 2023, 59(12): 78-88.