Digital Shop Floor Manufacturing Capability Modeling and Adaptive Scheduling in Human-cyber-physical Interconnected Environment

doi:10.3901/JME.2025.03.023

Abstract

Abstract: The development of Industry 5.0 presents higher requirements for the informatization, digitization, and intelligence of the manufacturing industry. To address the important challenges of the lack of traditional workshop manufacturing capacity organizational paradigm and intelligent scheduling technology, a digital workshop manufacturing capacity modeling and adaptive scheduling technology in the human-cyber-physical interconnected environment is proposed to achieve high-fidelity modeling of complex workshop manufacturing capacity and efficient scheduling of production resources. In order to effectively manage the production elements in the digital workshop, a digital workshop manufacturing capacity modeling technology that integrates the human-cyber-physical system is proposed. In addition, a deep reinforcement learning-driven adaptive scheduling algorithm (DRL-AS) is devised for the digital workshop, which models the flexible job shop scheduling problem in the form of heterogeneous directed acyclic graphs. Considering the complex coupling relationship between operations and machines, a multi-factor representation method based on hierarchical self-attention mechanism is designed to extract global features of the environmental state and assist the agent in making high-quality decisions. Proximal policy optimization (PPO) algorithm is used to train the proposed adaptive scheduling technology. Experimental results show that the scheduling performance and generalization performance of the proposed method are significantly better than those of the comparison algorithms.

Key words: human-cyber-physical system, flexible job-shop scheduling, deep reinforcement learning, manufacturing capability modeling, digital twin

CLC Number:

TH165
TP18

HU Bingtao, ZHONG Ruirui, FENG Yixiong, YANG Chen, WANG Tianyue, HONG Zhaoxi, TAN Jianrong. Digital Shop Floor Manufacturing Capability Modeling and Adaptive Scheduling in Human-cyber-physical Interconnected Environment[J]. Journal of Mechanical Engineering, 2025, 61(3): 23-39.

References

[1] 冯毅雄，钟锐锐，张志峰，等. 设计知识驱动的不规则多边形排样算法及应用[J]. 计算机集成制造系统，2023，29(2):593-603. FENG Yixiong，ZHONG Ruirui，ZHANG Zhifeng，et al. Irregular polygons nesting algorithm driven by design knowledge and application[J]. Computer Integrated Manufacturing System，2023，29(2):593-603.
[2] 张卫，王兴康，石涌江，等. 工业大数据驱动的智能制造服务系统构建技术[J]. 中国科学:技术科学，2023，53(7):1084-1096. ZHANG Wei，WANG Xingkang，SHI Yongjiang，et al. Construction technology of intelligent manufacturing service systems driven by industrial big data[J]. Scientia Sinica (Technologica)，2023，53(7):1084-1096.
[3] 胡炳涛，冯毅雄，刘继红，等. 面向“互联网+”定制产品的智能适应性设计研究[J]. 机械工程学报，2023，59(12):109-125. HU Bingtao，FENG Yixiong，LIU Jihong，et al. Research on intelligent adaptive design for “Internet+” customized products[J]. Journal of Mechanical Engineering，2023，59(12):109-125.
[4] ZHONG R，HU B，FENG Y，et al. Construction of human digital twin model based on multimodal data and its application in locomotion mode identification[J]. Chinese Journal of Mechanical Engineering，2023，36(126):1-12.
[5] 汪俊亮，高鹏捷，张洁，等. 制造大数据分析综述:内涵、方法、应用和趋势[J]. 机械工程学报，2023，59(12):1-16. WANG Junliang，GAO Pengjie，ZHANG Jie，et al. A review of manufacturing big data:Connotation，methodology，application，and trends[J]. Journal of Mechanical Engineering，2023，59(12):1-16.
[6] 陶飞，张萌，程江峰，等. 数字孪生车间——一种未来车间运行新模式[J]. 计算机集成制造系统，2017，23(1):1-9. TAO Fei，ZHANG Meng，CHENG Jiangfeng，et al. Digital twin workshop:A new paradigm for future workshop[J]. Computer Integrated Manufacturing Systems，2017，23(1):1-9.
[7] 谭建荣. 智能制造与机器人应用关键技术与发展趋势[J]. 机器人技术与应用，2017，3:18-19.
[8] ZHENG P，WANG H，SANG Z，et al. Smart manufacturing systems for Industry 4.0:Conceptual framework，scenarios，and future perspectives[J]. Frontiers of Mechanical Engineering，2018，13:137-150.
[9] ZHANG L，ZHOU L，REN L，et al. Modeling and simulation in intelligent manufacturing[J]. Computers in Industry，2019，112:103123.
[10] WANG T，PENG T，HU B，et al. Two-stage imbalanced learning-based quality prediction method for wheel hub assembly[J]. Advanced Engineering Informatics，2024，59:102309.
[11] 胡瑞淇，程辉，张执南. 基于表达式树的顺序柔性车间调度问题求解[J/OL]. 计算机集成制造系统:1-15[2023-10-24]. http://kns.cnki.net/kcms/detail/11.5946.tp. 20220317.1506.002.html. HU Ruiqi，CHENG Hui，ZHANG Zhinan. Solving job shop scheduling problem with flexible process sequence based on expression tree model[J]. Computer Integrated Manufacturing System:1-15[2023-10-24]. http://kns.cnki.net/kcms/detail/11.5946.tp.20220317.1506.002.html.
[12] 刘琼，刘嘉豪，刘佳良. 基于改进人工蜂群算法的预防性维修与柔性作业车间成组调度集成优化[J]. 机械工程学报，2023，59(12):89-96. LIU Qiong，LIU Jiahao，LIU Jialiang. Integrated optimization of preventive maintenance and flexible job shop group scheduling based on an ABC-AN algorithm[J]. Journal of Mechanical Engineering，2023，59(12):89-96.
[13] 谢法吾，李玲玲，李丽，等. 基于多目标混合进化算法的作业车间混排可变分批节能调度方法[J]. 中国机械工程，2023，34(13):1576-1588，1598. XIE Fawu，LI Lingling，LI Li，et al. Energy-efficient jop shop scheduling with variable lot splitting and sublots intermingling based on multi-objective hybrid evolutionary algorithm[J]. China Mechanical Engineering，2023，34(13):1576-1588，1598.
[14] 郭鹏，郝东辉，郑鹏，等. 考虑工人疲劳的双资源柔性作业车间调度优化[J]. 浙江大学学报(工学版)，2023，57(9):1804-1813. GUO Peng，HAO Donghui，ZHENG Peng，et al. Scheduling optimization of dual resource-constrained flexible job shop considering worker fatigue[J]. Journal of Zhejiang University (Engineering Science)，2023，57(9):1804-1813.
[15] ZHANG J，HE Z，CHAN W，et al. DeepMAG:Deep reinforcement learning with multi-agent graphs for flexible job shop scheduling[J]. Knowledge-Based Systems，2023，259:110083.
[16] YUAN E，CHENG S，WANG L，et al. Solving job shop scheduling problems via deep reinforcement learning[J]. Applied Soft Computing，2023，143:110436.
[17] LEI K，GUO P，ZHAO W，et al. A multi-action deep reinforcement learning framework for flexible Job-shop scheduling problem[J]. Expert Systems with Applications，2022，205:117796.
[18] SONG W，CHEN X，LI Q，et al. Flexible job-shop scheduling via graph neural network and deep reinforcement learning[J]. IEEE Transactions on Industrial Informatics，2022，19(2):1600-1610.
[19] WANG R，WANG G，SUN J，et al. Flexible job shop scheduling via dual attention network-based reinforcement learning[J]. IEEE Transactions on Neural Networks and Learning Systems，2023.
[20] LIU A，LUH P B.，SUN K，et al. Integrating machine learning and mathematical optimization for job shop scheduling[J]. IEEE Transactions on Automation Science and Engineering，2023.
[21] DESTOUET C，TLAHIG H，BETTAYEB B，et al. Flexible job shop scheduling problem under Industry 5.0:A survey on human reintegration，environmental consideration and resilience improvement[J]. Journal of Manufacturing Systems，2023，67:155-173.
[22] ZHONG R，HU B，HONG Z，et al. Human-Robot handover task intention recognition framework by fusing human digital twin and deep domain adaptation[J]. Journal of Engineering Design，2024，1-17.
[23] SKINNER W. Manufacturing-missing link in corporate strategy[J]. 1969.
[24] VASWANI A，SHAZEER N，PARMAR N，et al. Attention is all you need[J]. Advances in Neural Information Processing Systems，2017，30.
[25] CHEN R，LI W YANG H. A deep reinforcement learning framework based on an attention mechanism and disjunctive graph embedding for the job-shop scheduling problem[J]. IEEE Transactions on Industrial Informatics，2022，19(2):1322-1331.
[26] BRANDIMARTE P. Routing and scheduling in a flexible job shop by tabu search[J]. Annals of Operations research，1993，41(3):157-183.
[27] HURINK J，JURISCH B，THOLE M. Tabu search for the job-shop scheduling problem with multi-purpose machines[J]. Operations-Research-Spektrum，1994，15:205-215.
[28] 李峥峰，丁其聪，张东方，等. 改进离散麻雀搜索算法求解柔性作业车间调度问题[J]. 现代制造工程，2023(9):18-27. LI Zhengfeng，DING Qicong，ZHANG Dongfang，et al. Improved discrete sparrow search algorithm for flexible job-shop scheduling problem[J]. Modern Manufacturing Engineering，2023(9):18-27.
[29] 孟冠军，黄江涛，魏亚博. 混合白鲸优化算法求解柔性作业车间调度问题[J/OL]. 计算机工程与应用，1-10[2024-01-16]. http://kns.cnki.net/kcms/detail/11.2127.TP. 20231123.1409.015.html. MENG Guanjun，HUANG Jiangtao，WEI Yabo. Hybrid beluga whale optimization algorithm for flexible job shop scheduling problem[J]. Computer Engineering and Applications，1-10[2024-01-16]. http://kns.cnki.net/ kcms/detail/11.2127.TP.20231123.1409.015.html.
[30] 张凯，毕利，焦小刚. 集成强化学习算法的柔性作业车间调度问题研究[J]. 中国机械工程，2023，34(2):201-207. ZHANG Kai，BI Li，JIAO Xiaogang. Research on flexible job-shop scheduling problems with integrated reinforcement learning algorithm[J]. China Mechanical Engineering，2023，34(2):201-207.
[31] 吴昊泽，李艳武，谢辉. 改进PPO算法求解柔性作业车间调度问题[J/OL]. 计算机集成制造系统:1-20[2023-10-28]. http://kns.cnki.net/kcms/detail/11.5946.TP. 20230601.0942.006.html. WU Haoze，LI Yanwu，XIE Hui. Improved proximal policy optimization algorithm for solving flexible job shop scheduling problem[J/OL]. Computer Integrated Manufacturing System:1-20[2023-10-28]. http://kns. cnki.net/kcms/detail/11.5946.TP.20230601.0942.006.html.