Dynamic Scheduling Optimization of Island Assembly Lines Under Uncertain Disturbances by Multi-objective Deep Reinforcement Learning

doi:10.3901/JME.260229

Abstract

Abstract: With the rapid development of the new energy vehicle industry and the rise of diversified market demand and customization trends, an emerging island assembly mode has been introduced to address the lack of flexibility in the traditional automotive assembly line. Moreover, the frequent occurrence of uncertain events, such as emergency order insertion, severely restricts the stability and productivity of automotive final assembly in the actual assembly environment. Therefore, based on practical needs, dynamic scheduling optimization of island assembly lines under uncertain disturbances is conducted. First, a mixed-integer nonlinear programming model is formulated with the dual objectives of minimizing the maximum completion time and the order change index. Secondly, a multi-objective dueling double deep Q-network (MO-D3QN) is designed to solve this model. In this framework, state indicators and action scheduling rules are developed based on the features of assembly islands, assembly processes, assembly products, and production transportations in the island assembly scenario. Continuous immediate reward function components are constructed separately for dual optimization objectives, and reward aggregation is implemented by the weighted-sum scalarization method. Then, through the learning training for MO-D3QN network model to realize the selection of the optimized scheduling rules in different environment states. Finally, the computational experiment is conducted on three scaled instances. The results show that MO-D3QN outperforms the single scheduling rule, random selection strategy, and classical DQN, thereby verifying its effectiveness and competitiveness.

Key words: island assembly line, automotive assembly, uncertain disturbances, dynamic scheduling, multi-objective deep reinforcement learning

CLC Number:

TP18

HUANG Ming, HUANG Sihan, CHEN Jianpeng, DONG Wei, WANG Baicun, RUAN Bing, GAO Yunpeng, WANG Guoxin, YAN Yan. Dynamic Scheduling Optimization of Island Assembly Lines Under Uncertain Disturbances by Multi-objective Deep Reinforcement Learning[J]. Journal of Mechanical Engineering, 2026, 62(5): 74-87.

Add to citation manager EndNote|Reference Manager|ProCite|BibTeX|RefWorks

URL: http://www.cjmenet.com.cn/EN/10.3901/JME.260229

http://www.cjmenet.com.cn/EN/Y2026/V62/I5/74

References

[1] 工业和信息化部. 《汽车行业稳增长工作方案(2023—2024年)》[EB/OL]. https://www.gov.cn/zhengce/zhengceku/202309/content_6901733.htm. Ministry of Industry and Information Technology. 《Work Plan for Stable Growth in the Automotive Industry (2023-2024)》[EB/OL]. https://www.gov.cn/zhengce/zhengceku/202309/content_6901733.htm.
[2] LIU Yaqiong，SUN Shudong，SHEN Gaopan，et al. An auction-based approach for multi-agent uniform parallel machine scheduling with dynamic jobs arrival[J]. Engineering，2024，35:32-45.
[3] LI Yuxin，GU Wenbin，YUAN Minghai，et al. Real-time data-driven dynamic scheduling for flexible job shop with insufficient transportation resources using hybrid deep Q network[J]. Robotics and Computer-Integrated Manufacturing，2022，74:102283.
[4] LUO Shu. Dynamic scheduling for flexible job shop with new job insertions by deep reinforcement learning[J]. Applied Soft Computing，2020，91:106208.
[5] HUANG Ming，HUANG Sihan，DU Baigang，et al. Fuzzy superposition operation and knowledge-driven co-evolutionary algorithm for integrated production scheduling and vehicle routing problem with soft time windows and fuzzy travel times[J]. IEEE Transactions on Fuzzy Systems，2025，33(12):4152-4166.
[6] 李浩然，高亮，李新宇. 基于离散人工蜂群算法的多目标分布式异构零等待流水车间调度方法[J]. 机械工程学报，2023，59(2):291-306. LI Haoran，GAO Liang，LI Xinyu. Discrete artificial bee colony algorithm for multi-objective distributed heterogeneous no-wait flowshop scheduling problem[J]. Journal of Mechanical Engineering，2023，59(2):291-306.
[7] 吴秀丽，闫晓燕. 基于改进Q学习的可重入混合流水车间绿色动态调度[J]. 机械工程学报，2023，59(13):246-259. WU Xiuli，YAN Xiaoyan. An improved Q-learning algorithm to optimize green dynamic scheduling problem in a reentrant hybrid flow shop[J]. Journal of Mechanical Engineering，2023，59(13):246-259.
[8] LIU Youshan，FAN Jiaxin，ZHAO Linlin，et al. Integration of deep reinforcement learning and multi-agent system for dynamic scheduling of re-entrant hybrid flow shop considering worker fatigue and skill levels[J]. Robotics and Computer-Integrated Manufacturing，2023，84:102605.
[9] 贺俊杰，张洁，张朋，等. 基于多智能体强化学习的纺织面料染色车间动态调度方法[J]. 计算机集成制造系统，2023，29(1):62-74. HE Junjie，ZHANG Jie，ZHANG Peng，et al. Multi-agent reinforcement learning based textile dyeing workshop dynamic scheduling method[J]. Computer Integrated Manufacturing Systems，2023，29(1):62-74.
[10] SUN Mingyue，DING Jiyuchen，ZHAO Zhiheng，et al. Out-of-order execution enabled deep reinforcement learning for dynamic additive manufacturing scheduling[J]. Robotics and Computer-Integrated Manufacturing，2025，91:102841.
[11] 顾文斌，李育鑫，刘斯麒，等. 数据驱动的智慧车间实时调度方法研究[J]. 机械工程学报，2023，59:47-61. GU Wenbin，LI Yuxin，LIU Siqi，et al. Research on data-driven real-time scheduling method of smart workshop[J]. Journal of Mechanical Engineering，2023，59(12):47-61.
[12] LEI Kun，GUO Peng，WANG Yi，et al. Large-scale dynamic scheduling for flexible job-shop with random arrivals of new jobs by hierarchical reinforcement learning[J]. IEEE Transactions on Industrial Informatics，2024，20(1):1007-1018.
[13] HENGEL K，WAGNER A，RUSKOWSKI M. A dynamic multi-objective scheduling approach for gradient-based reinforcement learning[J]. IFAC-PapersOnLine，2024，58(19):49-54.
[14] YUE Lei，PENG Kai，DING Linshan，et al. Two-stage double deep Q-network algorithm considering external non-dominant set for multi-objective dynamic flexible job shop scheduling problems[J]. Swarm and Evolutionary Computation，2024，90:101660.
[15] LI Kaiwen，ZHANG Tao，WANG Rui. Deep reinforcement learning for multiobjective optimization[J]. IEEE Transactions on Cybernetics，2021，51(6):3103-3114.
[16] RIEDMILLER M，HAFNER R，LAMPE T，et al. Learning by playing solving sparse reward tasks from scratch[C]//Proceedings of the 35th International Conference on Machine Learning. Proceedings of Machine Learning Research; PMLR. 2018:4344-4353.
[17] MNIH V，KORAY K，SILVER D，et al. Playing atari with deep reinforcement learning[J]. ArXiv preprint，2013:arXiv:1312.5602.
[18] WANG Z，SCHAUL T，HESSEL M，et al. Dueling network architectures for deep reinforcement learning[C]//Proceedings of The 33rd International Conference on Machine Learning. Proceedings of Machine Learning Research; PMLR. 2016:1995-2003.
[19] HASSELT H V，GUE A，SILVER D. Deep reinforcement learning with double Q-learning[C]//Proceedings of the AAAI Conference on Artificial Intelligence，2016，30(1):2094-2100.
[20] HUANG Ming. Detailed data of the IAS-Dataset[EB/OL]. https://www.huangm.cn/cn/zip/IAS-Dataset.zip .
[21] HE Kaiming，ZHANG Xiangyu，REN Shaoqing，et al. Delving deep into rectifiers:Surpassing human-level performance on imagenet classification[C]//Proceedings of the 2015 IEEE International Conference on Computer Vision (ICCV). IEEE Computer Society. 2015:1026-1034.
[22] HUANG Ming，DU Baigang，GUO Jun. A hybrid collaborative framework for integrated production scheduling and vehicle routing problem with batch manufacturing and soft time windows[J]. Computers & Operations Research，2023，159:106346.