基于数字孪生和强化学习的智能生产线实时调度仿真优化方法

doi:10.3901/JME.260224

机械工程学报 ›› 2026, Vol. 62 ›› Issue (5): 12-25.doi: 10.3901/JME.260224

• 特邀专栏：信息驱动的总装拉动生产模式、技术及应用 • 上一篇

扫码分享

基于数字孪生和强化学习的智能生产线实时调度仿真优化方法

杨泽浩¹, 董威¹, 黄思翰^1,2, 阴艳超³, 董李扬⁴, 郑祖杰⁵

1. 北京理工大学机械与车辆学院北京 100081;
2. 北京理工大学知识与数据融合应用工业和信息化部重点实验室北京 100081;
3. 昆明理工大学机电工程学院昆明 650550;
4. 卡奥斯工业智能研究院(青岛)有限公司青岛 266000;
5. 上海航天精密机械研究所上海 201600

收稿日期:2025-08-05 修回日期:2025-12-05 发布日期:2026-04-23
作者简介:杨泽浩,男,2002年出生。主要研究方向为机器人加工、生产仿真。E-mail:2391646254@qq.com
董威,男,2002年出生。主要研究方向为智能仿真与优化。E-mail:1635611987@qq.com
黄思翰(通信作者),男,1991年出生,博士,特聘研究员,博士研究生导师。主要研究方向为具身智能可重构制造、人本智造、数字孪生。E-mail:hsh@bit.edu.cn
基金资助:
北京市自然科学基金重点研究专题(L243009)和国家自然科学基金(52405530)资助项目。

Real-time Scheduling Simulation Optimization Method for Smart Production Lines Based on Digital Twin and Reinforcement Learning

YANG Zehao¹, DONG Wei¹, HUANG Sihan^1,2, YIN Yanchao³, DONG Liyang⁴, ZHENG Zujie⁵

1. School of Mechanical Engineering, Beijing Institute of Technology, Beijing 100081;
2. Key Laboratory of Industry Knowledge & Data Fusion Technology and Application, Ministry of Industry and Information Technology, Beijing Institute of Technology, Beijing 100081;
3. Faculty of Mechanical and Electrical Engineering, Kunming University of Science and Technology, Kunming 650550;
4. COSMOPlat Industrial Intelligence Research Institute(Qingdao)Co., Ltd, Qingdao 266000;
5. Shanghai Spaceflight Precision Machinery Institute, Shanghai 201600

Received:2025-08-05 Revised:2025-12-05 Published:2026-04-23

摘要/Abstract

摘要： 生产调度一直是制造领域的研究热点，是生产线高效运行的节拍器。当前，随着智能制造的深入发展，生产调度智能化逐渐成为领域前沿。智能生产线动态生产过程中面临着生产任务变更、制造资源耦合等多源不确定扰动，如何兼顾生产调度效率和准确性是核心挑战。因此，提出了基于数字孪生和强化学习的智能生产线实时调度仿真优化方法。利用数字孪生技术构建生产要素的几何-功能-状态高保真模型，组装形成层次化、高保真虚拟生产仿真环境；设计改进Q-Learning算法建立生产调度优化智能体，通过三元组状态空间重构、多维度奖励函数及双重探索策略，突破传统算法维数灾难与鲁棒性不足难题；建立分层式执行控制架构实现数字孪生和智能体的深度融合，保障生产仿真过程感知-决策-执行闭环协同。某航天产品总装生产线验证结果表明，其他五种经典调度规则执行距离较本文方法增加6.38%到16.50%，显著提升制造资源协同效率。

关键词: 智能生产线, 生产调度, 数字孪生, 强化学习, 仿真优化

Abstract: Production scheduling remains a perpetual research hotspot in industry, serving as a critical metronome for the efficient operation of production lines. With the continuous evolution of intelligent manufacturing, smart production scheduling has emerged as a cutting-edge frontier. Multi-source stochastic disturbances, such as production task variations, the coupling of manufacturing resources, and others, pose a significant challenge in balancing scheduling efficiency and accuracy during dynamic production. To address this challenge, a real-time scheduling simulation optimization method based on digital twin (DT) and reinforcement learning (RL) is proposed. DT technology is used to construct high-fidelity models of production lines, establishing a hierarchical and high-fidelity virtual production simulation environment. An improved Q-Learning algorithm is developed to establish a scheduling optimization agent, incorporating triple state space reconstruction, a multi-dimensional reward function, and a dual exploration strategy to mitigate the curse of dimensionality and the robustness limitations inherent in traditional algorithms. Furthermore, a hierarchical execution control architecture is established based on perception-decision-execution loop throughout the production simulation process to achieve deep fusion between the DT and the intelligent simulation agent. A case study focusing on aerospace product final assembly line is provided to demonstrate the effectiveness of the proposed method. The result shows that the execution distances yielded by five other classical scheduling rules are 6.38% to 16.50% higher than those of the proposed method, signifying a substantial improvement in manufacturing resource collaborative efficiency.

Key words: smart production lines, production scheduling, digital twin, reinforcement learning, simulation optimization

中图分类号:

TG156

杨泽浩, 董威, 黄思翰, 阴艳超, 董李扬, 郑祖杰. 基于数字孪生和强化学习的智能生产线实时调度仿真优化方法[J]. 机械工程学报, 2026, 62(5): 12-25.

YANG Zehao, DONG Wei, HUANG Sihan, YIN Yanchao, DONG Liyang, ZHENG Zujie. Real-time Scheduling Simulation Optimization Method for Smart Production Lines Based on Digital Twin and Reinforcement Learning[J]. Journal of Mechanical Engineering, 2026, 62(5): 12-25.

导出引用管理器 EndNote|Reference Manager|ProCite|BibTeX|RefWorks

链接本文: http://www.cjmenet.com.cn/CN/10.3901/JME.260224

http://www.cjmenet.com.cn/CN/Y2026/V62/I5/12

参考文献

[1] HUANG S，WANG B，LI X，et al. Industry 5.0 and society 5.0—Comparison，complementation and co-evolution[J]. Journal of manufacturing systems，2022，64:424-428.
[2] HUANG S，WANG G，LEI D，et al. Toward digital validation for rapid product development based on digital twin:A framework[J]. The International Journal of Advanced Manufacturing Technology，2022，119(3):2509-2523.
[3] 王跃飞，王超，许于涛，等. 边-云协同下智能制造单元作业的数字孪生任务调度方法[J]. 机械工程学报，2024，60(6):137-152. WANG Yuefei，WANG Chao，XU Yutao，et al. Digital twin task scheduling method for intelligent manufacturing cell under edge-cloud collaboration[J]. Journal of Mechanical Engineering，2024，60(6):137-152.
[4] WANG D，WU H，ZHENG W，et al. A mixed-integer linear programming model for addressing efficient flexible flow shop scheduling problem with automatic guided vehicles consideration[J]. Applied Sciences，2025，15(6):3133-3141.
[5] LI W，HAN D，GAO L，et al. Integrated production and transportation scheduling method in hybrid flow shop[J]. Chinese Journal of Mechanical Engineering，2022，35(1):12.
[6] 孙媛，黄铭，刘合朋，等. 基于精益物流的光纤陀螺装配车间布局与生产调度集成优化[J]. 工业工程，2025，28(4):24-33. SUN Yuan，HUANG Ming，LIU Hepeng，et al. Integrated optimization of layout and scheduling in fiber optic gyroscope assembly workshops based on lean logistics[J]. Industrial Engineering Journal，2025，28(4):24-33.
[7] ZAN X，WU Z，GUO C，et al. A Pareto-based genetic algorithm for multi-objective scheduling of automated manufacturing systems[J]. Advances in Mechanical Engineering，2020，12(1):1687814019885294.
[8] LIU Z，ZHA J，YAN J，et al. An improved genetic algorithm with an overlapping strategy for solving a combination of order batching and flexible job shop scheduling problem[J]. Engineering Applications of Artificial Intelligence，2024，127:107321.
[9] 孙树栋，周新民，常昇博. 含私有信息的多代理作业车间协商调度算法[J]. 机械工程学报，2022，58(9):210-217. SUN Shudong，ZHOU Xinmin，CHANG Shengbo. Negotiation scheduling algorithm for multi-agent job shop with private information[J]. Journal of Mechanical Engineering，2022，58(9):210-217.
[10] ZHENG X，ZHANG C，AN Y，et al. A metaheuristic framework with experience reuse for dynamic multi-objective big data optimization[J]. Applied Sciences，2024，14(11):4878.
[11] 郭具涛，戴铮，张洁，等. 面向生产扰动的航天制造车间鲁棒性调度方法研究[J]. 工业工程，2025，28(2):69-77. GUO Jutao，DAI Zheng，ZHANG Jie，et al. Research on robust scheduling method for aerospace manufacturing workshop facing production disturbances[J]. Industrial Engineering Journal，2025，28(2):69-77.
[12] 黄思翰，彭志诚，朱启章，等. 面向工业元宇宙的人本智造系统数字孪生建模与分布式虚拟协作方法[J]. 机械工程学报，2025，61(15):385-398. HUANG Sihan，PENG Zhicheng，ZHU Qizhang，et al. Digital twin modeling and distributed virtual collaboration method for human-centric intelligent manufacturing system oriented to industrial metaverse[J]. Journal of Mechanical Engineering，2025，61(15):385-398.
[13] 林国义，郭慧妍，冷杰武，等. 数字孪生在工业工程领域应用的热点和趋势分析[J]. 工业工程，2024，27(6):13-25. LIN Guoyi，GUO Huiyan，LENG Jiewu，et al. Hotspots and trends analysis of digital twin application in industrial engineering[J]. Industrial Engineering Journal，2024，27(6):13-25.
[14] FENG X，WAN J. Digital twins for discrete manufacturing lines:A review[J]. Big Data and Cognitive Computing，2024，8(5):45.
[15] TAO，F，XIAO，B，QI，Q，et al. Digital twin modeling[J]. Journal of Manufacturing Systems，2022，64，372-389.
[16] 刘达新，王科，刘振宇，等. 基于数据融合与知识推理的机器人装配单元数字孪生建模方法研究[J]. 机械工程学报，2024，60(5):36-50. LIU Daxin，WANG Ke，LIU Zhenyu，et al. Research on digital twin modeling method of robotic assembly unit based on data fusion and knowledge reasoning[J]. Journal of Mechanical Engineering，2024，60(5):36-50.
[17] 顾文斌，李育鑫，刘斯麒，等. 数据驱动的智慧车间实时调度方法研究[J]. 机械工程学报，2023，59(12):47-61. GU Wenbin，LI Yuxin，LIU Silin，et al. Research on data-driven real-time scheduling method of smart workshop [J]. Journal of Mechanical Engineering，2023，59(12):47-61.
[18] 杨赓，周慧颖，王柏村. 数字孪生驱动的智能人机协作:理论、技术与应用[J]. 机械工程学报，2022，58(18):279-291. YANG Geng，ZHOU Huiying，WANG Bocun. Digital twin driven intelligent human-machine collaboration:Theory，technology and application[J]. Journal of Mechanical Engineering，2022，58(18):279-291.
[19] SANTOS R，PIQUEIRO H，DIAS R，et al. Transitioning trends into action:A simulation-based digital twin architecture for enhanced strategic and operational decision-making[J]. Computers & Industrial Engineering，2024，198:110616.
[20] XIAO B，ZHONG J，BAO X，et al. Digital twin-driven prognostics and health management for industrial assets[J]. Scientific Reports，2024，14(1):13443.
[21] KATYARA S，SHARMA S，DAMACHARLA P，et al. Benchmarking Sim2Real gap:High-fidelity digital twinning of agile manufacturing[J]. arXiv preprint arXiv:2409.10784，2024.
[22] LEE H，YANG H. Digital twin simulation and optimization of manufacturing process flows[C]//International Manufacturing Science and Engineering Conference. American Society of Mechanical Engineers，2023，87240:V002T07A013.
[23] KOBER C，ADOMAT V，AHANPANJEH M，et al. Digital twin fidelity requirements model for manufacturing[C]//Proceedings of the Conference on Production Systems and Logistics:CPSL 2022. Hannover:Publish-Ing.，2022:595-611.
[24] 沈倩，管在林，张正敏，等. 面向卷烟生产调度的集成产能过滤算法与仿真技术的优化框架[J]. 计算机集成制造系统，2022，28(5):1462-1471. SHEN Qian，GUAN Zailin，ZHANG Zhengmin，et al. Optimization framework integrating capacity filtering algorithm and simulation technology for cigarette production scheduling[J]. Computer Integrated Manufacturing Systems，2022，28(5):1462-1471.
[25] ZHANG M，WANG C，LI X，et al. Optimization of machine configuration and scheduling in the hybrid flow shop using a linear programming-driven evolutionary approach[J]. Robotics and Computer-Integrated Manufacturing，2025，95:103029.
[26] CAO X，YAO M，ZHANG Y，et al. Digital twin modeling and simulation optimization of transmission front and middle case assembly line[J]. Computer Modeling in Engineering & Sciences (CMES)，2024，139(3):3233-3253.
[27] DERLINI D，ANNISA S，LUBIS Z. Optimizing production scheduling in smart manufacturing systems using hybrid simulation-based multi-objective optimization[C]//Proceeding of International Conference on Science and Technology UISU. 2025:105-108.
[28] GRZNÁR P，PAPÁNEK L，MARČAN M，et al. Enhancing production efficiency through digital twin simulation scheduling[J]. Applied Sciences，2025，15(7):3637.
[29] 黄铭，黄思翰，陈建鹏，等. 基于多目标深度强化学习的不确定扰动下岛式装配线动态调度优化[J/OL]. 机械工程学报，1-14[2025-12-25]. https://link.cnki.net/urlid/11.2187.TH.20251223.1148.050. HUANG Ming，HUANG Sihan，CHEN Jianpeng，et al. Dynamic scheduling optimization of island assembly line under uncertain disturbance based on multi-objective deep reinforcement learning[J/OL]. Journal of Mechanical Engineering:1-14[2025-12-25].https://link.cnki.net/urlid/11.2187.TH.20251223.1148.050.
[30] LIU R，PIPLANI R，TORO C. Deep reinforcement learning for dynamic scheduling of a flexible job shop[J]. International Journal of Production Research，2022，60(13):4049-4069.
[31] 张中伟，李艺，高增恩，等. 基于深度强化学习的柔性作业车间节能调度研究[J]. 工业工程，2024，27(1):78-85，103. ZHANG Zhongwei，LI Yi，GAO Zengen，et al. Research on energy-saving scheduling of flexible job shop based on deep reinforcement learning[J]. Industrial Engineering Journal，2024，27(1):78-85，103.
[32] ZHANG M，LU Y，HU Y，et al. Dynamic scheduling method for job-shop manufacturing systems by deep reinforcement learning with proximal policy optimization[J]. sustainability，2022，14(9):5177.
[33] CHANG J，YU D，HU Y，et al. Deep reinforcement learning for dynamic flexible job shop scheduling with random job arrival[J]. Processes，2022，10(4):760.
[34] LIU R，PIPLANI R，TORO C. A deep multi-agent reinforcement learning approach to solve dynamic job shop scheduling problem[J]. Computers & Operations Research，2023，159:106294.
[35] LI H，ZHANG H，HE Z，et al. Solving integrated process planning and scheduling problem via graph neural network based deep reinforcement learning[J]. arXiv preprint arXiv:2409.00968，2024.
[36] DEVIDZE R，KAMALARUBAN P，SINGLA A. Exploration-guided reward shaping for reinforcement learning under sparse rewards[J]. Advances in Neural Information Processing Systems，2022，35:5829-5842.
[37] MOURTZIS D. Simulation in the design and operation of manufacturing systems:state of the art and new trends[J]. International Journal of Production Research，2020，58(7):1927-1949.
[38] QI Q，TAO F，HU T，et al. Enabling technologies and tools for digital twin[J]. Journal of Manufacturing Systems，2021，58:3-21.

基于数字孪生和强化学习的智能生产线实时调度仿真优化方法

Real-time Scheduling Simulation Optimization Method for Smart Production Lines Based on Digital Twin and Reinforcement Learning

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

本文评价

[1]	杨亮亮, 龚壮壮, 何西旺, 王沐晨, 闵强, 阚子云, 宋学官. 面向结构损伤识别的数字孪生建模[J]. 机械工程学报, 2026, 62(4): 328-341.
[2]	刘素艳, 董一林, 乔一鸣, 马增强. 基于改进蜣螂优化算法的轴承数字孪生模型的动态故障注入研究[J]. 机械工程学报, 2026, 62(3): 446-457.
[3]	王世博, 葛世荣, 管增伦, 周士林, 王丽杰, 李雪峰, 王赟, 袁晓明, 马广军. 面向数字孪生的采煤机滚筒实时截割载荷模型研究[J]. 机械工程学报, 2026, 62(3): 479-491.
[4]	刘海波, 邓平, 迟庆宇, 刘天然, 刘阔, 李特, 黄祖广, 刘行健, 薄其乐, STEVEN Y LIANG, 王永青. 智能加工中数据使能技术与应用[J]. 机械工程学报, 2026, 62(2): 407-444.
[5]	朱海华, 付泰然, 李霏, 刘长春, 蔡祺祥, 唐敦兵. 基于数字孪生的复杂产品装配物料过程齐套时间预测[J]. 机械工程学报, 2025, 61(8): 384-398.
[6]	李睿智, 陈悦敏, 闫纪红. 面向轨迹动态感知与自主决策的工业机器人数字孪生建模方法研究[J]. 机械工程学报, 2025, 61(7): 269-283.
[7]	胡炳涛, 钟锐锐, 冯毅雄, 杨晨, 王天跃, 洪兆溪, 谭建荣. 人-信息-物理互联环境下数字车间制造能力建模与自适应调度[J]. 机械工程学报, 2025, 61(3): 23-39.
[8]	马帅, 冷杰武, 陈祝云, 李巍华, 李波, 刘强. 基于数字孪生和深度迁移学习的电主轴热误差建模方法[J]. 机械工程学报, 2025, 61(3): 52-66.
[9]	杜国锋, 邵士博, 李尚霖, 林诚然, 曹政才. 融合视觉语言模型与近端策略优化算法的人形机器人步态切换方法[J]. 机械工程学报, 2025, 61(21): 204-212.
[10]	唐小林, 甘炯鹏, 张振果. 横纵向耦合跟车场景下基于多智能体深度强化学习的混合动力车队协同能量管理研究[J]. 机械工程学报, 2025, 61(2): 236-246.
[11]	王玉静, 李祎然, 康守强, 刘连胜, 李玉庆, 孙宇林. 基于数字孪生的不同工况下谐波减速器故障诊断方法[J]. 机械工程学报, 2025, 61(18): 12-26.
[12]	林歆悠, 代军, 曾松榕. 基于运动风险的强化学习换道决策方法研究[J]. 机械工程学报, 2025, 61(18): 252-266.
[13]	王柏村, 宋词, 苑艺修, 周慧颖, 鲍劲松, 黄思翰, 刘蔚然, 刘庭煜, 阮兵, 陶飞, 谢海波, 杨华勇. 面向人本智造的人体运动数字孪生研究与应用进展[J]. 机械工程学报, 2025, 61(15): 21-39.
[14]	乔非, 刘鹃, 王冬源, 丁晨, 施嘉璇, 王俊凯, 马玉敏. 工业5.0环境下面向生产调度的人本融合技术[J]. 机械工程学报, 2025, 61(15): 40-56.
[15]	黄思翰, 彭志诚, 朱启章, 王柏村, 张明睿, 马妮, 冷杰武, 郑湃, 敬石开, 王国新, 阎艳. 面向工业元宇宙的人本智造系统数字孪生建模与分布式虚拟协作方法[J]. 机械工程学报, 2025, 61(15): 385-398.