数字孪生驱动的智能人机协作：理论、技术与应用

doi:10.3901/JME.2022.18.279

机械工程学报 ›› 2022, Vol. 58 ›› Issue (18): 279-291.doi: 10.3901/JME.2022.18.279

扫码分享

数字孪生驱动的智能人机协作：理论、技术与应用

杨赓^1,2, 周慧颖^1,2, 王柏村^1,2

1. 浙江大学流体传动与机电系统国家重点实验室杭州 310027;
2. 浙江大学机械工程学院杭州 310027

收稿日期:2022-01-06 修回日期:2022-03-23 出版日期:2022-09-20 发布日期:2022-12-08
通讯作者: 王柏村(通信作者)，男，1990年出生，博士，研究员，博士研究生导师。主要研究方向为人-信息-物理系统(HCPS)，智能制造，系统优化与管理等。E-mail：baicunw@zju.edu.cn
作者简介:杨赓,男,1980年出生,博士,研究员,博士研究生导师。主要研究方向为人-信息-物理系统(HCPS),柔性电子,人机交互等;E-mail:yanggeng@zju.edu.cn;周慧颖,女,1996年出生,博士研究生。主要研究方向为人-信息-物理系统(HCPS),人机交互,可穿戴设备等;E-mail:huiyingzhou@zju.edu.cn
基金资助:
国家自然科学基金(51975513，51890884)、浙江省自然科学基金杰出青年基金(LR20E050003)、国家重点研发计划(2020YFB1713900)、宁波科技创新2025重大专项(2020Z022)、浙江大学百人计划科研启动(0020886)和浙江省软科学研究计划(2022C35040)资助项目。

Digital Twin-driven Smart Human-machine Collaboration: Theory, Enabling Technologies and Applications

YANG Geng^1,2, ZHOU Huiying^1,2, WANG Baicun^1,2

1. State Key Laboratory of Fluid Power and Mechatronic Systems, Zhejiang University, Hangzhou 310027;
2. School of Mechanical Engineering, Zhejiang University, Hangzhou 310027

Received:2022-01-06 Revised:2022-03-23 Online:2022-09-20 Published:2022-12-08

摘要/Abstract

摘要： 随着人-信息-物理系统(Human-cyber-physical systems, HCPS)的发展与演进，人机关系由人机共存、人机交互、人机合作逐渐发展到人机协作。与此同时，数字孪生等新兴使能技术的涌现推动人类生产生活的数字化、网络化和智能化发展，增强了人机协作的状态感知、认知分析和控制决策等能力，为面向人本智造的人机协作范式转变提供了支撑。基于HCPS理论，提出数字孪生驱动的智能人机协作的基本概念，通过对人机关系演变和人机协作定义进行探讨，阐明在HCPS语义下数字孪生驱动的智能人机协作具体内涵。针对目前人机协作模式中存在的问题，提出数字孪生驱动的智能人机协作框架体系，从人机物理实体、人机数字模型、连接与交互、智能协作决策服务四个主要组成部分进行阐述。在此基础上，围绕传感与集成、计算与分析、控制与执行三个方面分析智能人机协作的共性使能技术，包括可穿戴技术、柔性传感、人工智能、外骨骼等。讨论数字孪生驱动的智能人机协作在工业生产、建筑修造、医疗健康和人因工程等领域的典型应用，以期为推动人机协作范式转变、促进人机协作可持续发展提供参考。

关键词: 人机协作, 数字孪生, 人-信息-物理系统, 人本智造

Abstract: With the evolution of human-cyber-physical systems (HCPS), the relationships between human and machine evolved through different phases: from human-machine coexistence and human-machine interaction to human-machine cooperation and human-machine collaboration. Meanwhile, emerging technologies (e.g., digital twin) empower the industry and society with increased features of automation and intelligence, enhance the capabilities of perception, analyzing, control, and decision making. Those developments provide human-machine collaboration the foundation for paradigm shifting toward human-centric smart manufacturing. In this work, the concept of digital twin-driven smart human-machine collaboration is proposed based on the HCPS theory. By analyzing the evolutions of human-machine relationship and related definitions on human-machine collaboration, the connotation of smart human-machine collaboration is elaborated. The framework of digital twin-driven smart human-machine collaboration is proposed to address the current challenges in human-machine collaboration, from perspectives of physical entities, digital models, connection and interaction, and smart decision for collaborative service. The enabling technologies for smart human-machine collaboration are discussed from the aspects of sensing and integration, computation and analysis, control and execution, e.g., wearable technology, flexible sensing, artificial intelligence, exoskeleton. Furthermore, typical applications of smart human-machine collaboration are presented, including production, architecture construction, healthcare, and ergonomics. It is expected this work can provide a reference for facilitating paradigm shift of human-machine collaboration and the sustainable development of human-machine collaboration.

Key words: human-machine collaboration, digital twin, human-cyber-physical systems, human-centric smart manufacturing

中图分类号:

杨赓, 周慧颖, 王柏村. 数字孪生驱动的智能人机协作：理论、技术与应用[J]. 机械工程学报, 2022, 58(18): 279-291.

YANG Geng, ZHOU Huiying, WANG Baicun. Digital Twin-driven Smart Human-machine Collaboration: Theory, Enabling Technologies and Applications[J]. Journal of Mechanical Engineering, 2022, 58(18): 279-291.

参考文献

[1] ZHOU Ji，LI Peigen，ZHOU Yanhong，et al. Toward new-generation intelligent manufacturing[J]. Engineering. 2018，4 (1):11-20.
[2] ZHOU Ji，ZHOU Yanhong，WANG Baicun，et al. Human–cyber–physical systems (HCPSs) in the context of new-generation intelligent manufacturing[J]. Engineering，2019，5(4):624-636.
[3] 王柏村，臧冀原，屈贤明，等. 基于人-信息-物理系统(HCPS)的新一代智能制造研究[J]. 中国工程科学，2018，20(4):29-34.WANG Baicun，ZANG Jiyuan，QU Xianming，et al. Research on new-generation intelligent manufacturing based on human-cyber-physical systems[J]. Engineering Sciences，2018，20(4):29-34.
[4] 臧冀原，王柏村，孟柳，等. 智能制造的三个基本范式:从数字化制造、“互联网+”制造到新一代智能制造[J]. 中国工程科学，2018，20(4):13-18.ZANG Jiyuan，WANG Baicun，MENG Liu，et al. Brief analysis on three basic paradigms of intelligent manufacturing[J]. Engineering Sciences，2018，20(4):13-18.
[5] ANSARI F，HOLD P，KHOBREH M. A knowledge-based approach for representing jobholder profile toward optimal human–machine collaboration in cyber physical production systems [J]. CIRP Journal of Manufacturing Science and Technology，2020，28:87-106.
[6] 王柏村，黄思翰，易兵，等. 面向智能制造的人因工程研究与发展[J]. 机械工程学报，2020，56(16):240-253.WANG Baicun，HUANG Sihan，YI Bing，et al. State-of-art of human factors/ergonomics in intelligent manufacturing[J]. Journal of Mechanical Engineering，2020，56(16):240-253.
[7] 王柏村，薛塬，延建林，等. 以人为本的智能制造:理念、技术与应用[J].中国工程科学，2020，22(4):139-146.WANG Baicun，XUE Yuan，YAN Jianlin，et al. Human-centered intelligent manufacturing:overview and perspectives[J]. Strategic Study of CAE，2020，22(4):139-146.
[8] 陶飞，刘蔚然，张萌，等. 数字孪生五维模型及十大领域应用[J]. 计算机集成制造系统，2019，25(1):1-18.TAO Fei，LIU Weiran，ZHANG Meng，et al. Five-dimension digital twin model and its ten applications[J]. Computer Integrated Manufacturing Systems，2019，25(1):1-18.
[9] MCCAFFREY T，SPECTOR L. An approach to human–machine collaboration in innovation[J]. Artificial Intelligence for Engineering Design，Analysis and Manufacturing，2018，32(1):1-15.
[10] GRIEVES M，VICKERS J. Digital twin:mitigating unpredictable，undesirable emergent behavior in complex systems[M]. Berlin:Springer International Publishing，2017.
[11] 王柏村，易兵，刘振宇，等. HCPS视角下智能制造的发展与研究[J]. 计算机集成制造系统，2021，27(10):2749-2761.WANG Baicun，YI Bing，LIU Zhenyu，et al. Evolution and state-of-the-art of intelligent manufacturing from HCPS perspective[J]. Computer Integrated Manufacturing Systems，2021，27(10):2749-2761.
[12] SEBAN T M，BITONNEAU D，SALOTTI J M，et al. Human factors issues for the design of a cobotic system[C]// Proceedings of the AHFE 2016 International Conference on Human Factors in Robots and Unmanned Systems，Florida，USA，2016，499:375-385.
[13] WANG Lihui，GAO R，VÁNCZA J，et al. Symbiotic human-robot collaborative assembly[J]. CIRP Annals Manufacturing Technology，2019，68(2):701-726.
[14] HENTOUT A，AOUACHE M，MAOUDJ A，et al. Human-robot interaction in industrial collaborative robotics:A literature review of the decade 2008-2017[J]. Advanced Robotics，2019，33(15-16):764-799.
[15] KOLBEINSSON A，LAGERSTEDT E，LINDBLOM J. Foundation for a classification of collaboration levels for human-robot cooperation in manufacturing[J]. Production & Manufacturing Research，2019，7(1):448-471.
[16] HADDADIN S，CROFT E. Physical human-robot interaction[M]. Berlin:Springer International Publishing，2016.
[17] GALIN R，MESHCHERYAKOV R. Review on human-robot interaction during collaboration in a shared workspace[C]// International Conference on Interactive Collaborative Robotics，Istanbul，Turkey，2019，11659:63-74.
[18] BETTONI A，MONTINI E，RIGHI M. Mutualistic and adaptive human-machine collaboration based on machine learning in an injection moulding manufacturing line[J]. Procedia CIRP，2020，93:395-400.
[19] HAESEVOETS T，CREMER D D，DIERCKX K，et al. Human-machine collaboration in managerial decision making[J]. Computers in Human Behavior，2021，119:106730.
[20] MICHALOS G，KARAGIANNIS P，DIMITROPOULOS N，et al. The 21st century industrial robot:When tools become collaborators[M]. Berlin:Springer International Publishing，2021.
[21] MARUYAMA T，UESHIBA T，TADA M，et al. Digital twin-driven human robot collaboration using a digital human[J]. Sensors，2021，21(4):8266.
[22] BILBERG A，MALIK A A. Digital twin driven human-robot collaborative assembly[J]. CIRP Annals Manufacturing Technology，2019，68(1):499-502.
[23] ROETENBERG D，LUINGE H，SLYCKE P. Xsens MVN:Full 6dof human motion tracking using miniature inertial sensors[J]. Xsens Motion Technologies BV， 2009(1):1-9.
[24] PILATI F，FACCIO M，GAMBERI M，et al. Learning manual assembly through real-time motion capture for operator training with augmented reality[J]. Procedia Manufacturing，2020，45:189-195.
[25] XIE Li，YANG Geng，MANTYSALO M，et al. Heterogeneous integration of bio-sensing system-on-chip and printed electronics[J]. IEEE Journal on Emerging and Selected Topics in Circuits and Systems，2012，2(4):672-682.
[26] ROSA B M G，YANG Guangzhong. A flexible wearable device for measurement of cardiac，electrodermal，and motion parameters in mental healthcare applications[J]. IEEE Journal of Biomedical and Health Informatics，2019，23(6):2276-2285.
[27] AIVALIOTIS P，GEORGOULIAS K，ARKOULI Z，et al. Methodology for enabling digital twin using advanced physics-based modelling in predictive maintenance[J]. Procedia CIRP，2019，81:417-422.
[28] KUMARI S，JAIN R，KUMAR U. Defect identification in friction stir welding using continuous wavelet transform[J]. Journal of Intelligent Manufacturing，2019，3:483–494.
[29] PÉREZ L，JIMÉNEZ S R，RODRÍGUEZ N，et al. Digital twin and virtual reality based methodology for multi-robot manufacturing cell commissioning[J]. Applied Science，2020，10(10):3633.
[30] BARRICELLI B R，CASIRAGHI E，GLIOZZO J，et al. Human digital twin for fitness management[J]. IEEE Access，2020，8:26637-26664.
[31] SENGAN S，KUMAR K，SUBRAMANIYASWAMY V，et al. Cost-effective and efficient 3D human model creation and re-identification application for human digital twins[J]. Multimedia Tools for Digital Twin，2022，81:26839-26856.
[32] SUN Xuemin，BAO Jinsong，LI Jie，et al. A digital twin-driven approach for the assembly-commissioning of high precision products[J]. Robotics and Computer-Integrated Manufacturing，2020，61:101839.
[33] SIERLA S，KYRKI V，AARNIO P，et al. Automatic assembly planning based on digital product descriptions[J]. Computers in Industry，2018，97:34-46.
[34] HAVARD V，JEANNE B，LACOMBLEZ M，et al. Digital twin and virtual reality:A co-simulation environment for design and assessment of industrial workstations[J]. Production & Manufacturing Research，2019，7(1):472-489.
[35] MA Xin，CHENG Jiangfeng，QI Qinglin，et al. Artificial intelligence enhanced interaction in digital twin shop-floor[J]. Procedia CIRP，2021，100:858-863.
[36] DRÖDER K，BOBKA P，GERMANN T，et al. A machine learning-enhanced digital twin approach for human-robot-collaboration[J]. Procedia CIRP，2018，76:187-192.
[37] TU Xinyi，AUTIOSALO J，JADID A，et al. A mixed reality interface for a digital twin based crane[J]. Applied Science，2021，11(20):9480.
[38] HU Chunhua，FAN Weicun，ZEN Elan，et al. A digital twin-assisted real-time traffic data prediction method for 5g-enabled internet of vehicles[J]. IEEE Transactions on Industrial Informatics，2022，18(4):2811-2819.
[39] HUANG Sihan，WANG Guoxin，YAN Yan，et al. Blockchain-based data management for digital twin of product[J]. Journal of Manufacturing Systems，2020，54:361-371.
[40] Fraunhofer IPA，Drive systems and exoskeletons (2017)，https://www.ipa.fraunhofer.de/en/expertise/biomechatronic-systems/drive-systems-andexoskeletons.html (accessed on 21/02/2019).
[41] CONSTANTINESCU C，RUS R，RUSU C A，et al. Digital twins of exoskeleton-centered workplaces:Challenges and development methodology[J]. Procedia Manufacturing，2019，39:58-65.
[42] GHOSH A，SOTO D A P，VERES S M，et al. Human robot interaction for future remote manipulations in industry 4.0[C]// IFAC World Congress，Berlin，Germany. 2020，53(2):10223-10228.
[43] DUSCHAU W A，VON Z J，CAPREZ A. Path control:a method for patient-cooperative robot-aided gait rehabilitation[J]. IEEE Transactions on Neural Systems and Rehabilitation Engineering，2010，18(1):38-48.
[44] XU Yang，DING Cheng，SHU Xiaokang，et al. Shared control of a robotic arm using non-invasive brain–computer interface and computer vision guidance[J]. Robotics and Autonomous Systems，2019，115:121-129.
[45] LI Ming，OKAMURA A M. Recognition of operator motions for real-time assistance using virtual fixtures[C]// 11th Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems，Los Angeles，CA，USA，2003:125-131.
[46] OLIFF H，LIU Ying，KUMAR M，et al. Reinforcement learning for facilitating human-robot-interaction in manufacturing[J]. Journal of Manufacturing Systems，2020，56:326-340.
[47] KOUSI N，GKOURNELOS C，AIVALIOTIS S，et al. Digital twin for designing and reconfiguring human–robot collaborative assembly lines[J]. Applied Sciences，2021，11(10):4620.
[48] WANG Tian，LI Jiakun，KONG Zhaoning，et al. Digital twin improved via visual question answering for vision-language interactive mode in human–machine collaboration[J]. Journal of Manufacturing Systems，2021，58:261-269.
[49] WANG Xi，LIANG C J，MENASSA C C，et al. Interactive and immersive process-level digital twin for collaborative human–robot construction work[J]. Journal of Computing in Civil Engineering，2021，35(6):4021023.
[50] LIANG C J，MCGEE W，MENASSA C C，et al. Bi-directional communication bridge for state synchronization between digital twin simulations and physical construction robots[C]// Proceedings of the 37th the International Association for Automation and Robotics in Construction，Kitakyushu，Japan，2020，1480-1487.
[51] ZHOU Huiying，YANG Geng，LÜ Honghao，et al. IoT-enabled dual-arm motion capture and mapping for telerobotics in home care[J]. IEEE Journal of Biomedical and Health Informatics，2020，24(6):1541-1549.
[52] ZHOU Huiying，YANG Liu，LÜ Honghao，et al. Development of a synchronized human-robot-virtuality interaction system using cooperative robot and motion capture device[C]// IEEE/ASME International Conference on Advanced Intelligent Mechatronics (AIM)，Hong Kong，China，2019:329-334.
[53] YANG Geng，LÜ Honghao，ZHANG Zhiyu，et al. Keep healthcare workers safe:application of teleoperated robot in isolation ward for COVID-19 prevention and control[J]. Chinese Journal of Mechanical Engineering，2020，33:47.
[54] GRECO A，CATERINO M，FERA M，et al. Digital twin for monitoring ergonomics during manufacturing production[J]. Applied sciences，2020，10(21):7758.

数字孪生驱动的智能人机协作：理论、技术与应用

Digital Twin-driven Smart Human-machine Collaboration: Theory, Enabling Technologies and Applications

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

本文评价

[1]	张超, 周光辉, 李晶晶, 魏智博, 秦天宇. 面向航空复杂零件智能工艺规划的孪生工艺模型构建与应用研究[J]. 机械工程学报, 2024, 60(6): 32-43.
[2]	王跃飞, 王超, 许于涛, 孙睿, 肖锴, 王凯林. 边-云协同下智能制造单元作业的数字孪生任务调度方法[J]. 机械工程学报, 2024, 60(6): 137-152.
[3]	姜文涛, 刘晓明, 李培源, 陈海, 张煦松. 机械式真空快速切换开关机构多目标优化设计[J]. 机械工程学报, 2024, 60(6): 153-162.
[4]	刘达新, 王科, 刘振宇, 许嘉通, 谭建荣. 基于数据融合与知识推理的机器人装配单元数字孪生建模方法研究[J]. 机械工程学报, 2024, 60(5): 36-50.
[5]	石健, 刘冬, 王少萍. 基于数字孪生的机电液系统PHM关键技术综述[J]. 机械工程学报, 2024, 60(4): 66-81.
[6]	蒋周明矩, 熊异, 王柏村. 面向工业5.0的人机协作增材制造[J]. 机械工程学报, 2024, 60(3): 238-253.
[7]	杨亮亮, 来孝楠, 何西旺, 李鹏, 郭正刚, 宋学官. 面向飞机机翼数字孪生的在线加速度积分方法[J]. 机械工程学报, 2024, 60(2): 342-355.
[8]	喻惟刚, 文思捷, 李建军, 李心雨. 基于可供性推理的人机协作拆卸序列规划方法[J]. 机械工程学报, 2024, 60(17): 297-309.
[9]	陈健, 孟义兴, 袁慎芳, 徐秋慧, 王卉. 融合导波监测的搭接结构裂纹扩展寿命孪生预测[J]. 机械工程学报, 2024, 60(16): 34-42.
[10]	刘岳开, 王天杨, 褚福磊. 基于低延迟可解释性深度学习的复杂旋转机械关键部件知识嵌入与诊断方法研究[J]. 机械工程学报, 2024, 60(12): 107-115.
[11]	叶文昌, 郭必成, 邓朝晖, 王伟, 邹伶俐, 刘超, 尹刚刚, 姜峰. 刀具智能化关键技术的研究进展及发展趋势[J]. 机械工程学报, 2023, 59(23): 265-282.
[12]	侯正航, 何卫平, 王淑侠, 胡余彭, 肖一帆. 基于数字孪生的弧齿型导引头伺服机构动态装调技术研究[J]. 机械工程学报, 2023, 59(22): 433-443.
[13]	李宏坤, 魏代同, 陈玉刚, 许书本. 数字孪生驱动的旋转叶盘振动状态全景感知方法[J]. 机械工程学报, 2023, 59(21): 270-282.
[14]	谭建荣, 高铭宇, 徐敬华, 王林轩, 贾晨, 张树有, 王康. 数智化正向设计方法及其在制造装备与过程中的应用[J]. 机械工程学报, 2023, 59(19): 111-125.
[15]	周祖德, 姚碧涛. 数字制造的科学体系与技术架构[J]. 机械工程学报, 2023, 59(19): 126-151.