Digital Twin Modeling and Distributed Virtual Collaboration Method of Human-centric Smart Manufacturing System in the Context of Industrial Metauniverse

doi:10.3901/JME.2025.15.385

Abstract

Abstract: The industrial metaverse is a vertical application of metaverse technology in the industrial sector, which reshapes the production and manufacturing mode and industrial ecology. With the rise of Industry 5.0, the concept of human-centric smart manufacturing has gradually gained attention. The development of the industrial metaverse focusing on human-centric smart manufacturing has realized the deep integration of the two emerging concepts and has shown great research value and potential. Therefore, a digital twin modeling and distributed virtual collaboration method of human-centric smart manufacturing system in the context of industrial metauniverse is proposed. The aim is to address the challenges of rapid customization of human digital twin models and efficient collaboration among multi-human in a virtual space. A metamodel-based method for the rapid customization of human digital twins is developed, and virtual-real mapping and dynamic interaction between humans and their digital twins are realized with machine vision, which lays the foundation for the industrial metaverse. Next, a multi-human collaboration method for the metaverse space is proposed and a server-client distributed access framework is established to realize the integration and interaction of multi-human digital twins, which provides support for multi-person remote, real-time online, and effective collaboration. Finally, the effectiveness of the proposed approach is validated through a dual-human collaborative loading and unloading scenario in a human-centric smart manufacturing system.

Key words: industrial metaverse, human-centric smart manufacturing, human digital twin, human-robot collaboration, distributed virtual collaboration

CLC Number:

HUANG Sihan, PENG Zhicheng, ZHU Qizhang, WANG Bocun, ZHANG Mingrui, MA Ni, LENG Jiewu, ZHENG Pai, JING Shikai, WANG Guoxin, YAN Yan. Digital Twin Modeling and Distributed Virtual Collaboration Method of Human-centric Smart Manufacturing System in the Context of Industrial Metauniverse[J]. Journal of Mechanical Engineering, 2025, 61(15): 385-398.

References

[1] 黄思翰,王柏村,张美迪,等.面向人本智造的新一代操作工:参考架构、使能技术与典型场景[J].机械工程学报, 2022, 58(18):251-264. HUANG Sihan, WANG Baicun, ZHANG Meidi, et al. Operator 4.0 towards human-centric smart manufacturing:Framework, enabling technologies and typical scenarios[J]. Journal of Mechanical Engineering, 2022, 58(18):251-264.
[2] HUANG S, WANG B, LI X, et al. Industry 5.0 and society 5.0-Comparison, complementation and co-evolution[J]. Journal of Manufacturing Systems, 2022, 64:424-428.
[3] 王柏村,薛塬,延建林,等.以人为本的智能制造:理念、技术与应用[J].中国工程科学, 2020, 22(4):139-146. WANG Baicun, XUE Yuan, YAN Jianlin, et al. Human-centered intelligent manufacturing:Overview and perspectives[J]. Strategic Study of CAE, 2020, 22(4):139-146.
[4] WANG B, ZHENG P, YIN Y, et al. Toward human-centric smart manufacturing:A human-cyber-physical systems (HCPS) perspective[J]. Journal of Manufacturing Systems, 2022, 63:471-490.
[5] 王柏村.人本智造:面向新工业革命的制造模式[M].北京:电子工业出版社, 2023. WANG Baicun. Human-centric smart manufacturing[M]. Beijing:Publishing House of Electronics Industry, 2023.
[6] 马南峰,姚锡凡,陈飞翔,等.面向工业5.0的人本智造[J].机械工程学报, 2022, 58(18):88-102. MA Nanfeng, YAO Xinfan, CHEN Feixiang, et al. Human-centric smart manufacturing for Industry 5.0[J]. Journal of Mechanical Engineering, 2022, 58(18):88-102.
[7] WANG H, NING H, LING Y, et al. A Survey on the metaverse:The state-of-the-art, technologies, applications, and challenges[J]. IEEE Internet of Things Journal, 2023, 10(16):14671-14688.
[8] LEE J, KUNDU P. Integrated cyber-physical systems and industrial metaverse for remote manufacturing[J]. Manufacturing Letters, 2022, 34:12-15.
[9] YAO X, MA N, ZHANG J, et al. Enhancing wisdom manufacturing as industrial metaverse for industry and society 5.0[J]. Journal of Intelligent Manufacturing, 2022:1-21.
[10] LUCA G, ERWIN R, RENATO, Emerging research fields in safety and ergonomics in industrial collaborative robotics:A systematic literature review[J]. Robotics and Computer-Integrated Manufacturing, 2021, 67:101998.
[11] HALME R J, LANZ M, KÄMÄRÄINEN J, et al. Review of vision-based safety systems for human-robot collaboration[J]. Procedia CIRP, 2018, 72:111-116.
[12] LU C, GAO R, YIN L, et al. Human-robot collaborative scheduling in energy-efficient welding shop[J]. IEEE Transactions on Industrial Informatics, 2024, 20:963-971.
[13] LIU D, HUNAG Y, LIU Z, et al. A skeleton-based assembly action recognition method with feature fusion for human-robot collaborative assembly[J]. Journal of Manufacturing Systems, 2024, 76:553-566.
[14] 王文通,张智军,张铭洋.元宇宙关键技术、研究进展与应用综述[J].计算机科学, 2024, 51(12):2-11. WANG Wentong, ZHANG Zhijun ZHANGMingyang. Review of key technologies, research progress and applications of the metaverse[J]. Computer Science, 2024, 51(12):2-11.
[15] RAMASUBRAMANIAN AK, MATHEW R, KELLY M, et al. Digital twin for human-robot collaboration in manufacturing:Review and outlook[J]. Applied Sciences, 2022, 12(10):4811.
[16] TAO F, XIAO B, QI Q, et al. Digital twin modeling[J]. Journal of Manufacturing Systems, 2022, 64:372-389.
[17] 陶飞,张贺,戚庆林,等.数字孪生模型构建理论及应用[J].计算机集成制造系统, 2021, 27(1):1-15. TAO Fei, ZHANG He, QI Qiling, et al. Theory of digital twin modeling and its application[J]. Computer Integrated Manufacturing Systems, 2021, 27(1):1-15.
[18] 杨赓,周慧颖,王柏村.数字孪生驱动的智能人机协作:理论、技术与应用[J].机械工程学报, 2022, 58(18):279-291. YANG Geng, ZHOU Huiying, WANG Baicun, et al. Digital twin-driven smart human-machine collaboration:Theory, enabling technologies and applications[J]. Journal of Mechanical Engineering, 2022, 58(18):279-291.
[19] HE L, PAUL G, KAI L, et al. Method to integrate human simulation into gazebo for human-robot collaboration[J]. IOP Conference Series:Materials Science and Engineering, 2020, 825:013006.
[20] WANG B, ZHOU H, LI X, et al. Human digital twin in the context of Industry 5.0[J]. Robotics and Computer-Integrated Manufacturing, 2024, 85:102626.
[21] CHEN J, YI C, OKEGBILE S D, et al. Networking architecture and key supporting technologies for human digital twin in personalized healthcare:A comprehensive survey[J]. IEEE Communications Surveys&Tutorials, 2024, 26(1):706-746.
[22] 鲍劲松,张荣,李婕,等.面向人-机-环境共融的数字孪生协同技术[J].机械工程学报, 2022, 58(18):103-115. BAO Jisong, ZHANG Rong, LI Jie, et al. Digital-twin collaborative technology for human-robot-environment integration[J]. Journal of Mechanical Engineering, 2022, 58(18):103-115.
[23] MALIK A A, BREM A. Digital twins for collaborative robots:A case study in human-robot interaction[J]. Robotics and Computer-Integrated Manufacturing, 2021, 68:102092.
[24] BILBERG A, MALIK A A. Digital twin driven human-robot collaborative assembly[J]. CIRP Annals, 2019, 68(1):499-502.
[25] MALIK AA, BREM A. Digital twins of human robot collaboration in a production setting[J]. Procedia Manufacturing, 2018, 17:278-285.
[26] LIU S, WANG X, WANG L, Digital twin-enabled advance execution for human-robot collaborative assembly[J]. CIRP Annals, 2022, 71(1):25-28.
[27] YUAN G, LV F, SHI J, et al. Integrated optimisation of human-robot collaborative disassembly planning and adaptive evaluation driven by a digital twin[J]. International Journal of Production Research, 2024(28):1-9.
[28] WANG B, ZHOU H, YANG G, et al. Human digital twin (hdt) driven human-cyber-physical systems:Key technologies and applications[J]. Chinese Journal of Mechanical Engineering, 2022, 35(1):12-17.
[29] WANG B, ZHOU H, LI X, et al. Human digital twin in the context of industry 5.0[J]. Robotics and Computer-Integrated Manufacturing, 2024, 85:102626.
[30] LI C. Badminton motion capture with visual image detection of picking robotics[J]. International Journal of Advanced Robotic Systems, 2020:17(6):1-10.
[31] RHEE, W, KIM, Y G, LEE, J H, et al. Unconstrained lightweight control interface for robot-assisted minimally invasive surgery using MediaPipe framework and head-mounted display[J]. Virtual Reality, 2024, 28:114.
[32] AMJAD A, AZAM F, ANWAR M W, et al, A systematic review on the data interoperability of application layer protocols in Industrial IoT[J]. IEEE Access, 2021, 9:96528-96545.