面向认知赋能的人机协作：进展、挑战和展望

doi:10.3901/JME.2025.03.001

机械工程学报 ›› 2025, Vol. 61 ›› Issue (3): 1-22.doi: 10.3901/JME.2025.03.001

• 特邀专栏：人机联合认知赋能的高端装备设计、制造与运维 • 上一篇

扫码分享

面向认知赋能的人机协作：进展、挑战和展望

寇逸群¹, 杨晔¹, 刘颉², 胡友民¹, 李林³, 俞百川¹, 徐家和¹, 胡中旭¹, 史铁林¹

1. 华中科技大学机械科学与工程学院武汉 430074;
2. 华中科技大学土木与水利工程学院武汉 430074;
3. 国网电力科学研究院武汉南瑞有限责任公司武汉 430223

收稿日期:2024-08-29 修回日期:2024-12-04 发布日期:2025-03-12
作者简介:寇逸群，女，1998年出生，博士。主要研究方向为人机协作、认知建模。E-mail:kyq@hust.edu.cn;杨晔，男，1997年出生，博士。主要研究方向为数字孪生、知识图谱。E-mail:yangye2023@hust.edu.cn;刘颉，男，1988年出生，副教授。主要研究方向为水电能源系统安全运行管理专业领域，聚焦领域为知识图谱、工程物联感知、设备运维检修、虚拟人机交互、无人电站建设等。E-mail:jie_liu@hust.edu.cn;胡友民，男，1965年出生，教授。主要研究方向为数控装备、数字化工厂与虚拟制造、智能制造与控制、参数化驱动三维数字样机、机电设备状态监测、可靠性与安全性技术以及大型成套装备等。E-mail:youmhwh@hust.edu.cn;李林，男，1988年出生，工程师。主要研究方向为高电压技术、物资检测。E-mail:15827628272@163.com;俞百川，男，2001年出生，硕士。主要研究方向为数字孪生、培训系统。E-mail:yubaichuan@hust.edu.cn;徐家和，男，2002年出生，硕士。主要研究方向为具身智能。E-mail:xujiahe@hust.edu.cn;胡中旭(通信作者)，男，1993年出生，教授。主要研究方向为制造系统数字化与智能化、智能人机交互与协作、数字孪生驱动智能运维等。E-mail:zhongxu_hu@hust.edu.cn;史铁林，男，1963年出生，教授，国家级领军人才。主要研究方向为微系统与微制造、MEMS测试封装及可靠性、精密仪器与装备、状态监测与故障诊断、信号分析等。E-mail:tlshi@hust.edu.cn
基金资助:
国家自然科学基金(52205104)和国家重点研发计划(2023YFD2100905,2024YFE030008)资助项目。

Cognitive Empowerment for Human-robot Collaboration: Research Progress and Challenges

KOU Yiqun¹, YANG Ye¹, LIU Jie², HU Youmin¹, LI Lin³, YU Baichuan¹, XU Jiahe¹, HU Zhongxu¹, SHI Tielin¹

1. School of Mechanical Science & Engineering, Huazhong University of Science and Technology, Wuhan 430074;
2. School of Civil and Hydraulic Engineering, Huazhong University of Science and Technology, Wuhan 430074;
3. Nari Group Corporation, State Grid Electric Power Research Institute, Wuhan 430223

Received:2024-08-29 Revised:2024-12-04 Published:2025-03-12

摘要/Abstract

摘要： 在工业4.0向工业5.0的发展过程中，以人为本逐渐成为智能制造领域关注的焦点之一。当前的人机协作不仅强调要聚焦于技术的进步与效率的提升，更强调将人类的高阶认知思维与机器的计算能力相结合，实现认知赋能。基于此，梳理人机协作中认知赋能在交互感知、任务规划与执行、技能学习等关键领域的现有研究，揭示了多模态信息整合、任务推理、动态决策与技能知识表征的挑战。进一步，提出通过应用知识图谱构建的相关技术来支持人与其机器认知对齐的方法，以及通过应用知识图谱推理的相关技术来支持复杂环境下人机协作的任务优化和动态决策。在分析现有人机协作认知赋能研究局限性的基础上，展望未来智能制造环境下的深度认知协同的发展方向。

关键词: 人机协作, 认知赋能, 知识图谱, 知识表示, 多模态感知

Abstract: In the transition from Industry 4.0 to Industry 5.0, a human-centered approach has gradually emerged as a focal point in the field of smart manufacturing. Current human-machine collaboration not only emphasizes technological advancements and efficiency improvements but also stresses the integration of human higher-order cognitive thinking with machine computational capabilities to achieve cognitive empowerment. Based on this premise, this study reviews existing research on cognitive empowerment in human-machine collaboration, focusing on key areas such as interactive perception, task planning and execution, and skill learning. The challenges of multimodal information integration, task reasoning, dynamic decision-making, and skill knowledge representation are highlighted. Furthermore, methods are proposed to support human-machine cognitive using knowledge graph construction technologies, as well as to optimize tasks and facilitate dynamic decision-making in complex environments through the application of knowledge graph reasoning techniques. Building upon an analysis of the limitations in current research on cognitive empowerment in human-machine collaboration, this study also forecasts the future directions for deep cognitive collaboration within intelligent manufacturing environments.

Key words: human-machine collaboration, cognitive empowerment, knowledge graph, knowledge representation, multimodal perception

中图分类号:

TP182

寇逸群, 杨晔, 刘颉, 胡友民, 李林, 俞百川, 徐家和, 胡中旭, 史铁林. 面向认知赋能的人机协作：进展、挑战和展望[J]. 机械工程学报, 2025, 61(3): 1-22.

KOU Yiqun, YANG Ye, LIU Jie, HU Youmin, LI Lin, YU Baichuan, XU Jiahe, HU Zhongxu, SHI Tielin. Cognitive Empowerment for Human-robot Collaboration: Research Progress and Challenges[J]. Journal of Mechanical Engineering, 2025, 61(3): 1-22.

参考文献

[1] TAO F，ZHANG M. Digital twin shop-floor:A new shop- floor paradigm towards smart manufacturing[J]. IEEE Access，2017，5:20418-20427.
[2] GROSSE E H，SGARBOSSA F，BERLIN C，et al. Human-centric production and logistics system design and management:Transitioning from Industry 4.0 to Industry 5.0[J]. International Journal of Production Research，2023，61(22):7749-7759.
[3] LOU S，TAN R，ZHANG Y，et al. Personalized disassembly sequence planning for a human-robot hybrid disassembly cell[J]. IEEE Transactions on Industrial Informatics，2024:1-12.
[4] 娄山河，冯毅雄，胡炳涛，等. 人机认知协同的复杂装备概念设计:挑战、进展和展望[J]. 机械工程学报，2024，60(11):2-19. LOU Shanhe，FENG Yixiong，HU Bingtao，et al. Human- computer cognitive collaboration-driven conceptual design of complex equipment:Research progress and challenges[J]. Journal of Mechanical Engineering，2024，60(11):2-19.
[5] ZHANG X，YANG Y，ZHANG X，et al. A multi-level digital twin construction method of assembly line based on hybrid worker digital twin models[J]. Advanced Engineering Informatics，2024，62:102597.
[6] WANG B，ZHENG P，YIN Y，et al. Toward human- centric smart manufacturing:A human-cyber-physical systems (HCPS) perspective[J]. Journal of Manufacturing Systems，2022，63:471-490.
[7] DEMIR K A，DÖVEN G，SEZEN B. Industry 5.0 and human-robot co-working[J]. Procedia Computer Science，2019，158:688-695.
[8] LU YZHENG H，CHAND S，et al. Outlook on human- centric manufacturing towards Industry 5.0[J]. Journal of Manufacturing Systems，2022，62:612-627.
[9] 马南峰，姚锡凡，陈飞翔，等. 面向工业5.0的人本智造[J]. 机械工程学报，2022，58(18):88-102. MA Nanfeng，YAO Xifan，CHEN Feixiang，et al. Human-centric smart manufacturing for industry 5.0[J]. Journal of Mechanical Engineering，2022，58(18):88-102.
[10] LONGO F，PADOVANO A，UMBRELLO S. Value- oriented and ethical technology engineering in industry 5.0:a human-centric perspective for the design of the factory of the future[J]. Applied Sciences，2020，10(12):4182.
[11] ZHANG C，WANG Z，ZHOU G，et al. Towards new- generation human-centric smart manufacturing in Industry 5.0:A systematic review[J]. Advanced Engineering Informatics，2023，57:102121.
[12] ZHOU J，ZHOU Y，WANG B，et al. Human-cyber- physical systems (HCPSs)in the context of new-generation intelligent manufacturing[J]. Engineering，2019，5(4):624-636.
[13] LOU S，ZHANG Y，TAN R，et al. A human-cyber- physical system enabled sequential disassembly planning approach for a human-robot collaboration cell in Industry 5.0[J]. Robotics and Computer-Integrated Manufacturing，2024，87:102706.
[14] ZHANG C，ZHOU G，MA D，et al. A deep learning- enabled human-cyber-physical fusion method towards human-robot collaborative assembly[J]. Robotics and Computer-Integrated Manufacturing，2023，83:102571.
[15] 鲍劲松，张荣，李婕，等. 面向人-机-环境共融的数字孪生协同技术[J]. 机械工程学报，2022，58(18):103- 115. BAO Jinsong，ZHANG Rong，LI Jie，et al. Digital-twin collaborative technology for human-robot-environment integration[J]. Journal of Mechanical Engineering，2022，58(18):103-115.
[16] SONG C S，KIM Y K. The role of the human-robot interaction in consumers? acceptance of humanoid retail service robots[J]. Journal of Business Research，2022，146:489-503.
[17] YOGANATHAN V，OSBURG V S，KUNZ W H，et al. Check-in at the robo-desk:Effects of automated social presence on social cognition and service implications[J]. Tourism Management，2021，85:104309.
[18] LONG Y，WEI W，HUANG T，et al. Human-in-the-Loop embodied intelligence with interactive simulation environment for surgical robot learning[J]. IEEE Robotics and Automation Letters，2023，8(8):4441-4448.
[19] MENGALDO G，RENDA F，BRUNTON S L，et al. A concise guide to modelling the physics of embodied intelligence in soft robotics[J]. Nature Reviews Physics，2022，4(9):595-610.
[20] BOCKLISCH F，PACZKOWSKI G，ZIMMERMANN S，et al. Integrating human cognition in cyber-physical systems:A multidimensional fuzzy pattern model with application to thermal spraying[J]. Journal of Manufacturing Systems，2022，63:162-176.
[21] JAHANMAHIN R，MASOUD S，RICKLI J，et al. Human-robot interactions in manufacturing:A survey of human behavior modeling[J]. Robotics and Computer- Integrated Manufacturing，2022，78:102404.
[22] LI S，ZHENG P，LIU S，et al. Proactive human-robot collaboration:Mutual-cognitive，predictable，and self- organising perspectives[J]. Robotics and Computer- Integrated Manufacturing，2023，81:102510.
[23] ZHENG C，WANG K，GAO S，et al. Design of multi- modal feedback channel of human-robot cognitive interface for teleoperation in manufacturing[J]. Journal of Intelligent Manufacturing，2024.
[24] WANG X，ONG S K，NEE A Y C. Multi-modal augmented-reality assembly guidance based on bare-hand interface[J]. Advanced Engineering Informatics，2016，30(3):406-421.
[25] ROMERO D，BERNUS P，NORAN O，et al. The operator 4.0:human cyber-physical systems & adaptive automation towards human-automation symbiosis work systems[C]// Advances in Production Management Systems. Initiatives for a Sustainable World. Cham:Springer International Publishing，2016:677-686.
[26] HENSCHEL A，HORTENSIUS R，CROSS E S. Social cognition in the age of human-robot interaction[J]. Trends in Neurosciences，2020，43(6):373-384.
[27] WANG T，ZHENG P，LI S，et al. Multimodal human- robot interaction for human-centric smart manufacturing:A survey[J]. Advanced Intelligent Systems，2024，6(3):2300359.
[28] REN M，CHEN N，QIU H. Human-machine collaborative decision-making:An evolutionary roadmap based on cognitive intelligence[J]. International Journal of Social Robotics，2023，15(7):1101-1114.
[29] LEMAIGNAN S，WARNIER M，SISBOT E A，et al. Artificial cognition for social human-robot interaction:An implementation[J]. Artificial Intelligence，2017，247:45-69.
[30] CASALINO A，MESSERI C，POZZI M，et al. Operator awareness in human–robot collaboration through wearable vibrotactile feedback[J]. IEEE Robotics and Automation Letters，2018，3(4):4289-4296.
[31] HOC J M. From human-machine interaction to human- machine cooperation[J]. Ergonomics，2000，43(7):833-843.
[32] VINANZI S，CANGELOSI A，GOERICK C. The collaborative mind:Intention reading and trust in human- robot interaction[J]. iScience，2021，24(2):102130.
[33] HOFFMAN G，BHATTACHARJEE T，NIKOLAIDIS S. Inferring human intent and predicting human action in human-robot collaboration[J]. Annual Review of Control，Robotics，and Autonomous Systems，2024，7:73-95.
[34] 丛明，李金钟，刘冬，等. 目标导向的认知机制在机器人领域的应用综述[J]. 机械工程学报，2023，59(23):1-22. CONG Ming，LI Jinzhong，LIU Dong，et al. Review of the application of goal-directed cognitive mechanism in robotics[J]. Journal of Mechanical Engineering，2023，59(23):1-22.
[35] CAI J，GAO Z，GUO Y，et al. FedHIP:Federated learning for privacy-preserving human intention prediction in human-robot collaborative assembly tasks[J]. Advanced Engineering Informatics，2024，60:102411.
[36] LEE K H，BAEK S G，LEE H J，et al. Enhanced transparency for physical human-robot interaction using human hand impedance compensation[J]. IEEE/ASME Transactions on Mechatronics，2018，23(6):2662-2670.
[37] LI S，ZHANG X. Implicit intention communication in human-robot interaction through visual behavior studies[J]. IEEE Transactions on Human-Machine Systems，2017，47(4):437-448.
[38] ZHUANG Y，YAO S，MA C，et al. Admittance control based on EMG-driven musculoskeletal model improves the human-robot synchronization[J]. IEEE Transactions on Industrial Informatics，2019，15(2):1211-1218.
[39] LI J，LI Z，FENG Y，et al. Development of a human–robot hybrid intelligent system based on brain teleoperation and deep learning SLAM[J]. IEEE Transactions on Automation Science and Engineering，2019，16(4):1664-1674.
[40] PAUL S K，NICOLESCU M，NICOLESCU M. Enhancing human-robot collaboration through a multi- module interaction framework with sensor fusion:Object recognition，verbal communication，user of interest detection，gesture and gaze recognition[J]. Sensors，2023，23(13):5798.
[41] WONG C Y，VERGEZ L，SULEIMAN W. Vision- and tactile-based continuous multimodal intention and attention recognition for safer physical human-robot interaction[J]. IEEE Transactions on Automation Science and Engineering，2024，21(3):3205-3215.
[42] WANG W，LI R，CHEN Y，et al. Predicting human intentions in human-robot hand-over tasks through multimodal learning[J]. IEEE Transactions on Automation Science and Engineering，2022，19(3):2339-2353.
[43] WANG Y，WU Q，WANG S，et al. MI-EEG:Generalized model based on mutual information for EEG emotion recognition without adversarial training[J]. Expert Systems with Applications，2024，244:122777.
[44] JIANG C S，LIU Z T，WU M，et al. Efficient Facial expression recognition with representation reinforcement network and transfer self-training for human–machine interaction[J]. IEEE Transactions on Industrial Informatics，2023，19(9):9943-9952.
[45] KHAN M，GUEAIEB W，EL SADDIK A，et al. MSER:Multimodal speech emotion recognition using cross- attention with deep fusion[J]. Expert Systems with Applications，2024，245:122946.
[46] MENG T，SHOU Y，AI W，et al. A multi-message passing framework based on heterogeneous graphs in conversational emotion recognition[J]. Neurocomputing，2024，569:127109.
[47] LI W，ZENG G，ZHANG J，et al. CogEmoNet:A cognitive-feature-augmented driver emotion recognition model for smart cockpit[J]. IEEE Transactions on Computational Social Systems，2022，9(3):667-678.
[48] BUERKLE A，AL-YACOUB A，EATON W，et al. An incremental learning approach to detect muscular fatigue in human-robot collaboration[J]. IEEE Transactions on Human-Machine Systems，2023，53(3):520-528.
[49] LI Q，WANG Z，WANG W，et al. An adaptive time budget adjustment strategy based on a take-over performance model for passive fatigue[J]. IEEE Transactions on Human-Machine Systems，2022，52(5):1025-1035.
[50] MONTEIRO T G，SKOURUP C，ZHANG H. Using EEG for mental fatigue assessment:A comprehensive look into the current state of the art[J]. IEEE Transactions on Human-Machine Systems，2019，49(6):599-610.
[51] WANG L，LI H，YAO Y，et al. Smart cushion-based non-invasive mental fatigue assessment of construction equipment operators:A feasible study[J]. Advanced Engineering Informatics，2023，58:102134.
[52] MAHZOON H，OKAZAKI M，YOSHIKAWA Y，et al. Effect of the projection of robot’s talk information on the perception of communicating human[J]. Advanced Robotics，2021，35(20):1209-1222.
[53] ROY L，CROFT E A，KULIĆ D. Learning to communicate functional states with nonverbal expressions for improved human-robot collaboration[J]. IEEE Robotics and Automation Letters，2024，9(6):5393-5400.
[54] MULLEN J F，MOSIER J，CHAKRABARTI S，et al. Communicating inferred goals with passive augmented reality and active haptic feedback[J]. IEEE Robotics and Automation Letters，2021，6(4):8522-8529.
[55] AKHMETOV T，VAROL H A. An augmented reality- based warning system for enhanced safety in industrial settings[J]. IEEE Transactions on Industrial Informatics，2023，19(7):7966-7977.
[56] KASSEM K，UNGERBÖCK T，WINTERSBERGER P，et al. What is happening behind the wall? towards a better understanding of a hidden robot’s intent by multimodal cues[J]. Proc. ACM Hum.-Comput. Interact.，2022，6(MHCI):196:1-196:19.
[57] CHU M，CUI Z，ZHANG A，et al. Multisensory Fusion，haptic，and visual feedback teleoperation system under IoT framework[J]. IEEE Internet of Things Journal，2022，9(20):19717-19727.
[58] JOHANNSMEIER L，HADDADIN S. A Hierarchical human-robot interaction-planning framework for task allocation in collaborative industrial assembly processes[J]. IEEE Robotics and Automation Letters，2017，2(1):41-48.
[59] KRUEGER V，ROVIDA F，GROSSMANN B，et al. Testing the vertical and cyber-physical integration of cognitive robots in manufacturing[J]. Robotics and Computer-Integrated Manufacturing，2019，57:213-229.
[60] ZHANG R，LV Q，LI J，et al. A reinforcement learning method for human-robot collaboration in assembly tasks[J]. Robotics and Computer-Integrated Manufacturing，2022，73:102227.
[61] ESWARAN M，INKULU A kumar，TAMILARASAN K，et al. Optimal layout planning for human robot collaborative assembly systems and visualization through immersive technologies[J]. Expert Systems with Applications，2024，241:122465.
[62] LI Y，WANG Y，LI H，et al. Risk-aware decision making for human-robot collaboration with effective communication[J]. IEEE Transactions on Automation Science and Engineering，2023:1-0.
[63] PUPA A，VAN DIJK W，SECCHI C. A human-centered dynamic scheduling architecture for collaborative application[J]. IEEE Robotics and Automation Letters，2021，6(3):4736-4743.
[64] LEE M L，LIU W，BEHDAD S，et al. Robot-assisted disassembly sequence planning with real-time human motion prediction[J]. IEEE Transactions on Systems，Man，and Cybernetics:Systems，2023，53(1):438-450.
[65] ALIREZAZADEH S，ALEXANDRE L A. Dynamic task scheduling for human-robot collaboration[J]. IEEE Robotics and Automation Letters，2022，7(4):8699-8704.
[66] CRAMER M，KELLENS K，DEMEESTER E. Probabilistic decision model for adaptive task planning in human-robot collaborative assembly based on designer and operator intents[J]. IEEE Robotics and Automation Letters，2021，6(4):7325-7332.
[67] MESSERI C，BICCHI A，ZANCHETTIN A M，et al. A dynamic task allocation strategy to mitigate the human physical fatigue in collaborative robotics[J]. IEEE Robotics and Automation Letters，2022，7(2):2178-2185.
[68] HUANG W，ABBEEL P，PATHAK D，et al. Language models as zero-shot planners:Extracting actionable knowledge for embodied agents[C]//International conference on machine learning. PMLR，2022:9118-9147.
[69] DRIESS D，XIA F，SAJJADI M S M，et al. PaLM-E:An Embodied multimodal language model[J]. arXiv，2303.03378，2023.
[70] BROHAN A，BROWN N，CARBAJAL J，et al. RT-2:Vision-language-action models transfer web knowledge to robotic control[J]. arXiv，2307.15818，2023.
[71] SHIRAI K，BELTRAN-HERNANDEZ C C，HAMAYA M，et al. Vision-language interpreter for robot task planning[C]//2024 IEEE International Conference on Robotics and Automation (ICRA). 2024:2051-2058.
[72] XIE J，LIU S，WANG X. Framework for a closed-loop cooperative human cyber-physical system for the mining industry driven by VR and AR:MHCPS[J]. Computers & Industrial Engineering，2022，168:108050.
[73] WANG Z，BAI X，ZHANG S，et al. M-AR:A visual representation of manual operation precision in AR assembly[J]. International Journal of Human-Computer Interaction，2021，37(19):1799-1814.
[74] FANG W，FAN W，JI W，et al. Distributed cognition based localization for AR-aided collaborative assembly in industrial environments[J]. Robotics and Computer- Integrated Manufacturing，2022，75:102292.
[75] XIE J，LIU Y，WANG X，et al. A new XR-based human- robot collaboration assembly system based on industrial metaverse[J]. Journal of Manufacturing Systems，2024，74:949-964.
[76] VANNESTE P，DEKEYSER K，ULLAURI L A P，et al. Towards tailored cognitive support in augmented reality assembly work instructions[J]. Journal of Computer Assisted Learning，2024，40(2):797-811.
[77] FOGLI D，GARGIONI L，GUIDA G，et al. A hybrid approach to user-oriented programming of collaborative robots[J]. Robotics and Computer-Integrated Manufacturing，2022，73:102234.
[78] KROEMER O，NIEKUM S，KONIDARIS G. A review of robot learning for manipulation:Challenges，representations，and algorithms[J]. Journal of Machine Learning Research，2021，22(30):1-82.
[79] SCHAAL S. Learning from demonstration[C]//Advances in Neural Information Processing Systems，1996，9:1-6.
[80] SCHOU C，ANDERSEN R S，CHRYSOSTOMOU D，et al. Skill-based instruction of collaborative robots in industrial settings[J]. Robotics and Computer-Integrated Manufacturing，2018，53:72-80.
[81] YIN Y，ZHENG P，LI C，et al. Enhancing human-guided robotic assembly:AR-assisted DT for skill-based and low- code programming[J]. Journal of Manufacturing Systems，2024，74:676-689.
[82] ZHAO T Z，KUMAR V，LEVINE S，et al. Learning fine-grained bimanual manipulation with low-cost hardware[EB/OL]. (2023-04-23)[2024-10-09]. https://arxiv.org/abs/2304.13705v1.
[83] ZENG C，YANG C，CHENG H，et al. Simultaneously encoding movement and sEMG-based stiffness for robotic skill learning[J]. IEEE Transactions on Industrial Informatics，2021，17(2):1244-1252.
[84] WU H，YAN W，XU Z，et al. A framework of robot skill learning from complex and long-horizon tasks[J]. IEEE Transactions on Automation Science and Engineering，2022，19(4):3628-3638.
[85] WANG W，LI R，CHEN Y，et al. Facilitating human- robot collaborative tasks by teaching-learning-collaboration from human demonstrations[J]. IEEE Transactions on Automation Science and Engineering，2019，16(2):640-653.
[86] HU Z，ZHENG Y，PAN J. Grasping living objects with adversarial behaviors using inverse reinforcement learning[J]. IEEE Transactions on Robotics，2023，39(2):1151-1163.
[87] APOLINARSKA A A，PACHER M，LI H，et al. Robotic assembly of timber joints using reinforcement learning[J]. Automation in Construction，2021，125:103569.
[88] SUN H，ZHANG T，HAN J，et al. A fast transfer reinforcement learning model for transferring force-based human speed adjustment skills to robots for collaborative assembly posture alignment[J]. Advanced Engineering Informatics，2024，62:102836.
[89] GUPTA A，SAVARESE S，GANGULI S，et al. Embodied intelligence via learning and evolution[J]. Nature Communications，2021，12(1):5721.
[90] GUZMAN L，MORELLAS V，PAPANIKOLOPOULOS N. Robotic embodiment of human-like motor skills via reinforcement learning[J]. IEEE Robotics and Automation Letters，2022，7(2):3711-3717.
[91] KOMULAINEN T M，SANNERUD A R. Learning transfer through industrial simulator training:Petroleum industry case[J]. Cogent Education，2018，5(1):1554790.
[92] SHAYESTEH S，OJHA A，LIU Y，et al. Human-robot teaming in construction:Evaluative safety training through the integration of immersive technologies and wearable physiological sensing[J]. Safety Science，2023，159:106019.
[93] ESTRADA J，PAHEDING S，YANG X，et al. Deep- learning-incorporated augmented reality application for engineering lab training[J]. APPLIED SCIENCES- BASEL，2022，12(10):5159.
[94] EHRLINGER L，WÖSS W. Towards a definition of knowledge graphs[C]//International Conference on Semantic Systems. 2016:1-6.
[95] 黄恒琪，于娟，廖晓，等. 知识图谱研究综述[J]. 计算机系统应用，2019，28(6):1-12. HUANG Hengqi，YU Juan，LIAO Xiao，et al. Review on knowledge graphs[J]. Computer Systems Applications，2019，28(6):1-12.
[96] WANG G，HE J. A Bibliometric analysis of recent developments and trends in knowledge graph research (2013-2022)[J]. IEEE Access，2024，12:32005-32013.
[97] GUO H，LIU K. Hierarchical ontology graph for solving semantic issues in decision support systems[C]// Proceedings of the 21st International Conference on Enterprise Information Systems (ICEIS)，Setubal:Scitepress，2019:483-487.
[98] ZHAO G，ZHANG X. Domain-specific ontology concept extraction and hierarchy extension[C]//Proceedings of the 2nd International Conference on Natural Language Processing and Information Retrieval. New York，NY，USA:Association for Computing Machinery，2018:60-64.
[99] YUAN J，LI H. Research on the standardization model of data semantics in the knowledge graph construction of oil&gas industry[J]. Computer Standards & Interfaces，2023，84:103705.
[100] CHEN Y，FU L，QIU J，et al. Semantic based logistics data exchange model[C]//2022 7th International Conference on Intelligent Computing and Signal Processing (ICSP). 2022:928-933.
[101] KRÖTZSCH M，THOST V. Ontologies for knowledge graphs:Breaking the rules[C]//The Semantic Web-ISWC 2016:15th International Semantic Web Conference，Kobe，Japan，October 17-21，2016，Proceedings，Part I 15. Springer International Publishing，2016:376-392.
[102] KUMAR N，KUMAR M，SINGH M. Automated ontology generation from a plain text using statistical and NLP techniques[J]. International Journal of System Assurance Engineering and Management，2016，7(S1):282-293.
[103] KUWANA A，OBA A，SAWAI R，et al. Automatic taxonomy classification by pretrained language model[J]. Electronics，2021，10(21):2656.
[104] XU D，CHEN W，PENG W，et al. Large language models for generative information extraction:A survey[J]. Frontiers of Computer Science，2024，18(6):186357.
[105] QIAO B，ZOU Z，HUANG Y，et al. A joint model for entity and relation extraction based on BERT[J]. Neural Computing and Applications，2022，34(5):3471-3481.
[106] DAGDELEN J，DUNN A，LEE S，et al. Structured information extraction from scientific text with large language models[J]. Nature Communications，2024，15(1):1418.
[107] ZAFARIAN A，ROKNI A，KHADIVI S，et al. Semi-supervised learning for named entity recognition using weakly labeled training data[C]//2015 The International Symposium on Artificial Intelligence and Signal Processing (AISP). 2015:129-135.
[108] XU T，GUO C，DU L，et al. A method for traditional chinese medicine knowledge graph dynamic construction[C]// Proceedings of the 5th International Conference on Big Data Technologies. New York，NY，USA:Association for Computing Machinery，2022:196-202.
[109] PAN Z，SU C，DENG Y，et al. Video2Entities:A computer vision-based entity extraction framework for updating the architecture，engineering and construction industry knowledge graphs[J]. Automation in Construction，2021，125:103617.
[110] ZHENG C，FENG J，FU Z，et al. Multimodal relation extraction with efficient graph alignment[C]// Proceedings of the 29th ACM International Conference on Multimedia. New York，NY，USA:Association for Computing Machinery，2021:5298-5306.
[111] BORDES A，USUNIER N，GARCIA-DURAN A，et al. Translating embeddings for modeling multi-relational data[C]//Advances in Neural Information Processing Systems:Vol. 26. Curran Associates，Inc.，2013.
[112] ASMARA S M，SAHABUDIN N A，NADIAH ISMAIL N S，et al. A review of knowledge graph embedding methods of transe，transh and transr for missing links[C]// 2023 IEEE 8th International Conference On Software Engineering and Computer Systems (ICSECS). 2023:470-475.
[113] JIA Y，WANG Y，JIN X，et al. Knowledge graph embedding:A locally and temporally adaptive translation- based approach[J]. ACM Trans. Web，2017，12(2):8:1-8:33.
[114] LI Y，ZHU C. TransE-MTP:A new representation learning method for knowledge graph embedding with multi-translation principles and transe[J]. Electronics，2024，13(16):3171.
[115] SUN Z，DENG Z H，NIE J Y，et al. RotatE:Knowledge graph embedding by relational rotation in complex space[EB/OL]. (2019-02-26)[2024-08-28]. https://arxiv.org/abs/1902.10197v1.
[116] DETTMERS T，MINERVINI P，STENETORP P，et al. Convolutional 2D knowledge graph embeddings[J]. Proceedings of the AAAI Conference on Artificial Intelligence，2018，32(1):1811-1818.
[117] ZHAO X，ZHANG J，CAO Y，et al. Spatio-temporal knowledge embedding method considering the lifecycle of geographical entities[J]. International Journal of Applied Earth Observation and Geoinformation，2024，131:103967.
[118] YANG B，YIH W，HE X，et al. Embedding entities and relations for learning and inference in knowledge bases[EB/OL]. (2014-12-20)[2024-08-28]. https://arxiv.org/abs/1412.6575v4.
[119] TROUILLON T，WELBL J，RIEDEL S，et al. Complex embeddings for simple link prediction[C]//Proceedings of The 33rd International Conference on Machine Learning. PMLR，2016:2071-2080.
[120] MA Y，TRESP V，DAXBERGER E A. Embedding models for episodic knowledge graphs[J]. Journal of Web Semantics，2019，59:100490.
[121] WANG J，ZHANG W，CHEN X，et al. 3DRTE:3D rotation embedding in temporal knowledge graph[J]. IEEE Access，2020，8:207515-207523.
[122] XU X，LIN K，YANG Y，et al. Joint Feature Synthesis and embedding:Adversarial cross-modal retrieval revisited[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence，2022，44(6):3030-3047.
[123] LI Y，SU H，QI C R，et al. Joint embeddings of shapes and images via CNN image purification[J]. ACM Trans. Graph.，2015，34(6):234:1-234:12.
[124] MITHUN N C，LI J，METZE F，et al. Joint embeddings with multimodal cues for video-text retrieval[J]. International Journal of Multimedia Information Retrieval，2019，8(1):3-18.
[125] NAWAZ S，CALEFATI A，JANJUA M K，et al. Learning fused representations for large-scale multimodal classification[J]. IEEE Sensors Letters，2019，3(1):1-4.
[126] HUANG F，ZHANG X，XU J，et al. Network embedding by fusing multimodal contents and links[J]. Knowledge- Based Systems，2019，171:44-55.
[127] HAZARIKA D，PORIA S，ZIMMERMANN R，et al. Conversational transfer learning for emotion recognition[J]. Information Fusion，2021，65:1-12.
[128] YUAN Y，QU A. High-order joint embedding for multi-level link prediction[J]. Journal of the American Statistical Association，2023，118(543):1692-1706.
[129] GAO J，LYU T，XIONG F，et al. Predicting the survival of cancer patients with multimodal graph neural network[J]. IEEE/ACM Transactions on Computational Biology and Bioinformatics，2022，19(2):699-709.
[130] TAO Z，WEI Y，WANG X，et al. MGAT:multimodal graph attention network for recommendation[J]. Information Processing & Management，2020，57(5):102277.
[131] ZENG Z，CHENG Q，SI Y. Logical rule-based knowledge graph reasoning:A comprehensive survey[J]. Mathematics，2023，11(21):4486.
[132] 田玲，张谨川，张晋豪，等. 知识图谱综述——表示、构建、推理与知识超图理论[J]. 计算机应用，2021，41(8):2161-2186. TIAN Ling，ZHANG Jinchuan，ZHANG Jinhao，et al. Knowledge graph survey:Representation，construction，reasoning and knowledge hypergraph theory[J]. Journal of Computer Applications，2021，41(8):2161-2186.
[133] MAYR M，AHMAD F，CHATZILYGEROUDIS K，et al. Combining planning，reasoning and reinforcement learning to solve industrial robot tasks[J]. arXiv，2212.03570，2022.
[134] THIKE P H，XU Z，CHENG Y，et al. Materials failure analysis utilizing rule-case based hybrid reasoning method[J]. Engineering Failure Analysis，2019，95:300-311.
[135] SHI X，TIAN X，GU J，et al. A hybrid approach of case- and rule-based reasoning to assembly sequence planning[J]. The International Journal of Advanced Manufacturing Technology，2023，127(1):221-236.
[136] SMITH F，PROIETTI M. Rule-based behavioral reasoning on semantic business processes[C]//ICAART (2). 2013:130-143.
[137] ZHANG L，YU M，GAO T，et al. MCMH:Learning multi-chain multi-hop rules for knowledge graph reasoning[EB/OL]. (2020-10-05)[2024-08-28]. https://arxiv.org/abs/2010.01735v1.
[138] WANG J，HUANG Y. Logical reasoning over knowledge graphs in vector space[C]//Second International Symposium on Computer Applications and Information Systems (ISCAIS 2023):Vol. 12721. SPIE，2023:272-278.
[139] TANG X，ZHU S，LIANG Y，et al. RulE:Knowledge graph reasoning with rule embedding[C]//Findings of the Association for Computational Linguistics ACL 2024. 2024:4316-4335.
[140] NICKEL M，MURPHY K，TRESP V，et al. A review of relational machine learning for knowledge graphs[J]. Proceedings of the IEEE，2016，104(1):11-33.
[141] MA T，LV S，HUANG L，et al. HiAM:A hierarchical attention based model for knowledge graph multi-hop reasoning[J]. Neural Networks，2021，143:261-270.
[142] ZHANG Y，JIN L，LI X，et al. Edge-aware graph neural network for multi-hop path reasoning over knowledge base[J]. Computational Intelligence and Neuroscience，2022，2022(1):4734179.
[143] TRIVEDI R，DAI H，WANG Y，et al. Know-evolve: Deep temporal reasoning for dynamic knowledge graphs[C]//Proceedings of the 34th International Conference on Machine Learning. PMLR，2017:3462-3471.
[144] GUO T，YANG Q，WANG C，et al. Knowledge navigator:Leveraging large language models for enhanced reasoning over knowledge graph[J]. Complex & Intelligent Systems，2024，10(5):7063-7076.
[145] ABU-RASHEED H，WEBER C，FATHI M. Knowledge graphs as context sources for llm-based explanations of learning recommendations[J]. arXiv，2403.03008，2024.
[146] LIU H，ZHOU S，CHEN C，et al. Dynamic knowledge graph reasoning based on deep reinforcement learning[J]. Knowledge-Based Systems，2022，241:108235.
[147] BAI L，YU W，CHEN M，et al. Multi-hop reasoning over paths in temporal knowledge graphs using reinforcement learning[J]. Applied Soft Computing，2021，103:107144.
[148] KIM Y J，KWAK B，KIM Y，et al. Modularized transfer learning with multiple knowledge graphs for zero-shot commonsense reasoning[J]. arXiv，2206. 03715.
[149] WU Y，ZHOU J. CogTrans:A cognitive transfer learning-based self-attention mechanism architecture for knowledge graph reasoning[J]. Proceedings of the Annual Meeting of the Cognitive Science Society，2023，45(45):1893-1898.

面向认知赋能的人机协作：进展、挑战和展望

Cognitive Empowerment for Human-robot Collaboration: Research Progress and Challenges

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

本文评价

[1]	蒋周明矩, 熊异, 王柏村. 面向工业5.0的人机协作增材制造[J]. 机械工程学报, 2024, 60(3): 238-253.
[2]	丁满, 孙鸣宇, 张新新, 冯光宇. 基于知识超图的产品色彩情感设计[J]. 机械工程学报, 2024, 60(17): 339-348.
[3]	喻惟刚, 文思捷, 李建军, 李心雨. 基于可供性推理的人机协作拆卸序列规划方法[J]. 机械工程学报, 2024, 60(17): 297-309.
[4]	余隋怀, 王鹏超, 王磊, 初建杰, 敖卿, 侯幸刚. 基于知识工程的产品设计研究综述[J]. 机械工程学报, 2024, 60(13): 216-234.
[5]	吴秀丽, 马隆洲, 向东, 唐英. 面向退役机电产品全生命周期的知识图谱统一建模方法研究[J]. 机械工程学报, 2023, 59(7): 52-67.
[6]	吴其林, 赵韩, 陈晓飞, 赵雅婷. 多臂协作机器人技术与应用现状及发展趋势[J]. 机械工程学报, 2023, 59(15): 1-16.
[7]	从靖晨, 项忠霞, 李心雨, 董良, 李巨韬, 陈俊贤. 基于知识图谱的智能产品服务系统交互设计研究[J]. 机械工程学报, 2023, 59(11): 94-105.
[8]	杨赓, 周慧颖, 王柏村. 数字孪生驱动的智能人机协作：理论、技术与应用[J]. 机械工程学报, 2022, 58(18): 279-291.
[9]	鲍劲松, 张荣, 李婕, 陆玉前, 彭涛. 面向人-机-环境共融的数字孪生协同技术[J]. 机械工程学报, 2022, 58(18): 103-115.
[10]	马南峰, 姚锡凡, 陈飞翔, 俞鸿均, 王柯赛. 面向工业5.0的人本智造[J]. 机械工程学报, 2022, 58(18): 88-102.
[11]	张栋豪, 刘振宇, 郏维强, 刘惠, 谭建荣. 知识图谱在智能制造领域的研究现状及其应用前景综述[J]. 机械工程学报, 2021, 57(5): 90-113.
[12]	荆泓玮, 朱延河, 赵思恺, 张清华, 赵杰. 外肢体机器人研究现状及发展趋势[J]. 机械工程学报, 2020, 56(7): 1-9.
[13]	严洪森. 新的先进制造模式知识表示方法[J]. , 2006, 42(10): 80-90.
[14]	周福章;徐军. 智能控制知识分类表示与获取新方法[J]. , 1998, 34(2): 73-78.
[15]	赵克;叶尚辉. 适用于并行工程的资源模型[J]. , 1997, 33(1): 15-21，3.