Cognitive Empowerment for Human-robot Collaboration: Research Progress and Challenges

doi:10.3901/JME.2025.03.001

Abstract

Abstract: In the transition from Industry 4.0 to Industry 5.0, a human-centered approach has gradually emerged as a focal point in the field of smart manufacturing. Current human-machine collaboration not only emphasizes technological advancements and efficiency improvements but also stresses the integration of human higher-order cognitive thinking with machine computational capabilities to achieve cognitive empowerment. Based on this premise, this study reviews existing research on cognitive empowerment in human-machine collaboration, focusing on key areas such as interactive perception, task planning and execution, and skill learning. The challenges of multimodal information integration, task reasoning, dynamic decision-making, and skill knowledge representation are highlighted. Furthermore, methods are proposed to support human-machine cognitive using knowledge graph construction technologies, as well as to optimize tasks and facilitate dynamic decision-making in complex environments through the application of knowledge graph reasoning techniques. Building upon an analysis of the limitations in current research on cognitive empowerment in human-machine collaboration, this study also forecasts the future directions for deep cognitive collaboration within intelligent manufacturing environments.

Key words: human-machine collaboration, cognitive empowerment, knowledge graph, knowledge representation, multimodal perception

CLC Number:

TP182

KOU Yiqun, YANG Ye, LIU Jie, HU Youmin, LI Lin, YU Baichuan, XU Jiahe, HU Zhongxu, SHI Tielin. Cognitive Empowerment for Human-robot Collaboration: Research Progress and Challenges[J]. Journal of Mechanical Engineering, 2025, 61(3): 1-22.

References

[1] TAO F，ZHANG M. Digital twin shop-floor:A new shop- floor paradigm towards smart manufacturing[J]. IEEE Access，2017，5:20418-20427.
[2] GROSSE E H，SGARBOSSA F，BERLIN C，et al. Human-centric production and logistics system design and management:Transitioning from Industry 4.0 to Industry 5.0[J]. International Journal of Production Research，2023，61(22):7749-7759.
[3] LOU S，TAN R，ZHANG Y，et al. Personalized disassembly sequence planning for a human-robot hybrid disassembly cell[J]. IEEE Transactions on Industrial Informatics，2024:1-12.
[4] 娄山河，冯毅雄，胡炳涛，等. 人机认知协同的复杂装备概念设计:挑战、进展和展望[J]. 机械工程学报，2024，60(11):2-19. LOU Shanhe，FENG Yixiong，HU Bingtao，et al. Human- computer cognitive collaboration-driven conceptual design of complex equipment:Research progress and challenges[J]. Journal of Mechanical Engineering，2024，60(11):2-19.
[5] ZHANG X，YANG Y，ZHANG X，et al. A multi-level digital twin construction method of assembly line based on hybrid worker digital twin models[J]. Advanced Engineering Informatics，2024，62:102597.
[6] WANG B，ZHENG P，YIN Y，et al. Toward human- centric smart manufacturing:A human-cyber-physical systems (HCPS) perspective[J]. Journal of Manufacturing Systems，2022，63:471-490.
[7] DEMIR K A，DÖVEN G，SEZEN B. Industry 5.0 and human-robot co-working[J]. Procedia Computer Science，2019，158:688-695.
[8] LU YZHENG H，CHAND S，et al. Outlook on human- centric manufacturing towards Industry 5.0[J]. Journal of Manufacturing Systems，2022，62:612-627.
[9] 马南峰，姚锡凡，陈飞翔，等. 面向工业5.0的人本智造[J]. 机械工程学报，2022，58(18):88-102. MA Nanfeng，YAO Xifan，CHEN Feixiang，et al. Human-centric smart manufacturing for industry 5.0[J]. Journal of Mechanical Engineering，2022，58(18):88-102.
[10] LONGO F，PADOVANO A，UMBRELLO S. Value- oriented and ethical technology engineering in industry 5.0:a human-centric perspective for the design of the factory of the future[J]. Applied Sciences，2020，10(12):4182.
[11] ZHANG C，WANG Z，ZHOU G，et al. Towards new- generation human-centric smart manufacturing in Industry 5.0:A systematic review[J]. Advanced Engineering Informatics，2023，57:102121.
[12] ZHOU J，ZHOU Y，WANG B，et al. Human-cyber- physical systems (HCPSs)in the context of new-generation intelligent manufacturing[J]. Engineering，2019，5(4):624-636.
[13] LOU S，ZHANG Y，TAN R，et al. A human-cyber- physical system enabled sequential disassembly planning approach for a human-robot collaboration cell in Industry 5.0[J]. Robotics and Computer-Integrated Manufacturing，2024，87:102706.
[14] ZHANG C，ZHOU G，MA D，et al. A deep learning- enabled human-cyber-physical fusion method towards human-robot collaborative assembly[J]. Robotics and Computer-Integrated Manufacturing，2023，83:102571.
[15] 鲍劲松，张荣，李婕，等. 面向人-机-环境共融的数字孪生协同技术[J]. 机械工程学报，2022，58(18):103- 115. BAO Jinsong，ZHANG Rong，LI Jie，et al. Digital-twin collaborative technology for human-robot-environment integration[J]. Journal of Mechanical Engineering，2022，58(18):103-115.
[16] SONG C S，KIM Y K. The role of the human-robot interaction in consumers? acceptance of humanoid retail service robots[J]. Journal of Business Research，2022，146:489-503.
[17] YOGANATHAN V，OSBURG V S，KUNZ W H，et al. Check-in at the robo-desk:Effects of automated social presence on social cognition and service implications[J]. Tourism Management，2021，85:104309.
[18] LONG Y，WEI W，HUANG T，et al. Human-in-the-Loop embodied intelligence with interactive simulation environment for surgical robot learning[J]. IEEE Robotics and Automation Letters，2023，8(8):4441-4448.
[19] MENGALDO G，RENDA F，BRUNTON S L，et al. A concise guide to modelling the physics of embodied intelligence in soft robotics[J]. Nature Reviews Physics，2022，4(9):595-610.
[20] BOCKLISCH F，PACZKOWSKI G，ZIMMERMANN S，et al. Integrating human cognition in cyber-physical systems:A multidimensional fuzzy pattern model with application to thermal spraying[J]. Journal of Manufacturing Systems，2022，63:162-176.
[21] JAHANMAHIN R，MASOUD S，RICKLI J，et al. Human-robot interactions in manufacturing:A survey of human behavior modeling[J]. Robotics and Computer- Integrated Manufacturing，2022，78:102404.
[22] LI S，ZHENG P，LIU S，et al. Proactive human-robot collaboration:Mutual-cognitive，predictable，and self- organising perspectives[J]. Robotics and Computer- Integrated Manufacturing，2023，81:102510.
[23] ZHENG C，WANG K，GAO S，et al. Design of multi- modal feedback channel of human-robot cognitive interface for teleoperation in manufacturing[J]. Journal of Intelligent Manufacturing，2024.
[24] WANG X，ONG S K，NEE A Y C. Multi-modal augmented-reality assembly guidance based on bare-hand interface[J]. Advanced Engineering Informatics，2016，30(3):406-421.
[25] ROMERO D，BERNUS P，NORAN O，et al. The operator 4.0:human cyber-physical systems & adaptive automation towards human-automation symbiosis work systems[C]// Advances in Production Management Systems. Initiatives for a Sustainable World. Cham:Springer International Publishing，2016:677-686.
[26] HENSCHEL A，HORTENSIUS R，CROSS E S. Social cognition in the age of human-robot interaction[J]. Trends in Neurosciences，2020，43(6):373-384.
[27] WANG T，ZHENG P，LI S，et al. Multimodal human- robot interaction for human-centric smart manufacturing:A survey[J]. Advanced Intelligent Systems，2024，6(3):2300359.
[28] REN M，CHEN N，QIU H. Human-machine collaborative decision-making:An evolutionary roadmap based on cognitive intelligence[J]. International Journal of Social Robotics，2023，15(7):1101-1114.
[29] LEMAIGNAN S，WARNIER M，SISBOT E A，et al. Artificial cognition for social human-robot interaction:An implementation[J]. Artificial Intelligence，2017，247:45-69.
[30] CASALINO A，MESSERI C，POZZI M，et al. Operator awareness in human–robot collaboration through wearable vibrotactile feedback[J]. IEEE Robotics and Automation Letters，2018，3(4):4289-4296.
[31] HOC J M. From human-machine interaction to human- machine cooperation[J]. Ergonomics，2000，43(7):833-843.
[32] VINANZI S，CANGELOSI A，GOERICK C. The collaborative mind:Intention reading and trust in human- robot interaction[J]. iScience，2021，24(2):102130.
[33] HOFFMAN G，BHATTACHARJEE T，NIKOLAIDIS S. Inferring human intent and predicting human action in human-robot collaboration[J]. Annual Review of Control，Robotics，and Autonomous Systems，2024，7:73-95.
[34] 丛明，李金钟，刘冬，等. 目标导向的认知机制在机器人领域的应用综述[J]. 机械工程学报，2023，59(23):1-22. CONG Ming，LI Jinzhong，LIU Dong，et al. Review of the application of goal-directed cognitive mechanism in robotics[J]. Journal of Mechanical Engineering，2023，59(23):1-22.
[35] CAI J，GAO Z，GUO Y，et al. FedHIP:Federated learning for privacy-preserving human intention prediction in human-robot collaborative assembly tasks[J]. Advanced Engineering Informatics，2024，60:102411.
[36] LEE K H，BAEK S G，LEE H J，et al. Enhanced transparency for physical human-robot interaction using human hand impedance compensation[J]. IEEE/ASME Transactions on Mechatronics，2018，23(6):2662-2670.
[37] LI S，ZHANG X. Implicit intention communication in human-robot interaction through visual behavior studies[J]. IEEE Transactions on Human-Machine Systems，2017，47(4):437-448.
[38] ZHUANG Y，YAO S，MA C，et al. Admittance control based on EMG-driven musculoskeletal model improves the human-robot synchronization[J]. IEEE Transactions on Industrial Informatics，2019，15(2):1211-1218.
[39] LI J，LI Z，FENG Y，et al. Development of a human–robot hybrid intelligent system based on brain teleoperation and deep learning SLAM[J]. IEEE Transactions on Automation Science and Engineering，2019，16(4):1664-1674.
[40] PAUL S K，NICOLESCU M，NICOLESCU M. Enhancing human-robot collaboration through a multi- module interaction framework with sensor fusion:Object recognition，verbal communication，user of interest detection，gesture and gaze recognition[J]. Sensors，2023，23(13):5798.
[41] WONG C Y，VERGEZ L，SULEIMAN W. Vision- and tactile-based continuous multimodal intention and attention recognition for safer physical human-robot interaction[J]. IEEE Transactions on Automation Science and Engineering，2024，21(3):3205-3215.
[42] WANG W，LI R，CHEN Y，et al. Predicting human intentions in human-robot hand-over tasks through multimodal learning[J]. IEEE Transactions on Automation Science and Engineering，2022，19(3):2339-2353.
[43] WANG Y，WU Q，WANG S，et al. MI-EEG:Generalized model based on mutual information for EEG emotion recognition without adversarial training[J]. Expert Systems with Applications，2024，244:122777.
[44] JIANG C S，LIU Z T，WU M，et al. Efficient Facial expression recognition with representation reinforcement network and transfer self-training for human–machine interaction[J]. IEEE Transactions on Industrial Informatics，2023，19(9):9943-9952.
[45] KHAN M，GUEAIEB W，EL SADDIK A，et al. MSER:Multimodal speech emotion recognition using cross- attention with deep fusion[J]. Expert Systems with Applications，2024，245:122946.
[46] MENG T，SHOU Y，AI W，et al. A multi-message passing framework based on heterogeneous graphs in conversational emotion recognition[J]. Neurocomputing，2024，569:127109.
[47] LI W，ZENG G，ZHANG J，et al. CogEmoNet:A cognitive-feature-augmented driver emotion recognition model for smart cockpit[J]. IEEE Transactions on Computational Social Systems，2022，9(3):667-678.
[48] BUERKLE A，AL-YACOUB A，EATON W，et al. An incremental learning approach to detect muscular fatigue in human-robot collaboration[J]. IEEE Transactions on Human-Machine Systems，2023，53(3):520-528.
[49] LI Q，WANG Z，WANG W，et al. An adaptive time budget adjustment strategy based on a take-over performance model for passive fatigue[J]. IEEE Transactions on Human-Machine Systems，2022，52(5):1025-1035.
[50] MONTEIRO T G，SKOURUP C，ZHANG H. Using EEG for mental fatigue assessment:A comprehensive look into the current state of the art[J]. IEEE Transactions on Human-Machine Systems，2019，49(6):599-610.
[51] WANG L，LI H，YAO Y，et al. Smart cushion-based non-invasive mental fatigue assessment of construction equipment operators:A feasible study[J]. Advanced Engineering Informatics，2023，58:102134.
[52] MAHZOON H，OKAZAKI M，YOSHIKAWA Y，et al. Effect of the projection of robot’s talk information on the perception of communicating human[J]. Advanced Robotics，2021，35(20):1209-1222.
[53] ROY L，CROFT E A，KULIĆ D. Learning to communicate functional states with nonverbal expressions for improved human-robot collaboration[J]. IEEE Robotics and Automation Letters，2024，9(6):5393-5400.
[54] MULLEN J F，MOSIER J，CHAKRABARTI S，et al. Communicating inferred goals with passive augmented reality and active haptic feedback[J]. IEEE Robotics and Automation Letters，2021，6(4):8522-8529.
[55] AKHMETOV T，VAROL H A. An augmented reality- based warning system for enhanced safety in industrial settings[J]. IEEE Transactions on Industrial Informatics，2023，19(7):7966-7977.
[56] KASSEM K，UNGERBÖCK T，WINTERSBERGER P，et al. What is happening behind the wall? towards a better understanding of a hidden robot’s intent by multimodal cues[J]. Proc. ACM Hum.-Comput. Interact.，2022，6(MHCI):196:1-196:19.
[57] CHU M，CUI Z，ZHANG A，et al. Multisensory Fusion，haptic，and visual feedback teleoperation system under IoT framework[J]. IEEE Internet of Things Journal，2022，9(20):19717-19727.
[58] JOHANNSMEIER L，HADDADIN S. A Hierarchical human-robot interaction-planning framework for task allocation in collaborative industrial assembly processes[J]. IEEE Robotics and Automation Letters，2017，2(1):41-48.
[59] KRUEGER V，ROVIDA F，GROSSMANN B，et al. Testing the vertical and cyber-physical integration of cognitive robots in manufacturing[J]. Robotics and Computer-Integrated Manufacturing，2019，57:213-229.
[60] ZHANG R，LV Q，LI J，et al. A reinforcement learning method for human-robot collaboration in assembly tasks[J]. Robotics and Computer-Integrated Manufacturing，2022，73:102227.
[61] ESWARAN M，INKULU A kumar，TAMILARASAN K，et al. Optimal layout planning for human robot collaborative assembly systems and visualization through immersive technologies[J]. Expert Systems with Applications，2024，241:122465.
[62] LI Y，WANG Y，LI H，et al. Risk-aware decision making for human-robot collaboration with effective communication[J]. IEEE Transactions on Automation Science and Engineering，2023:1-0.
[63] PUPA A，VAN DIJK W，SECCHI C. A human-centered dynamic scheduling architecture for collaborative application[J]. IEEE Robotics and Automation Letters，2021，6(3):4736-4743.
[64] LEE M L，LIU W，BEHDAD S，et al. Robot-assisted disassembly sequence planning with real-time human motion prediction[J]. IEEE Transactions on Systems，Man，and Cybernetics:Systems，2023，53(1):438-450.
[65] ALIREZAZADEH S，ALEXANDRE L A. Dynamic task scheduling for human-robot collaboration[J]. IEEE Robotics and Automation Letters，2022，7(4):8699-8704.
[66] CRAMER M，KELLENS K，DEMEESTER E. Probabilistic decision model for adaptive task planning in human-robot collaborative assembly based on designer and operator intents[J]. IEEE Robotics and Automation Letters，2021，6(4):7325-7332.
[67] MESSERI C，BICCHI A，ZANCHETTIN A M，et al. A dynamic task allocation strategy to mitigate the human physical fatigue in collaborative robotics[J]. IEEE Robotics and Automation Letters，2022，7(2):2178-2185.
[68] HUANG W，ABBEEL P，PATHAK D，et al. Language models as zero-shot planners:Extracting actionable knowledge for embodied agents[C]//International conference on machine learning. PMLR，2022:9118-9147.
[69] DRIESS D，XIA F，SAJJADI M S M，et al. PaLM-E:An Embodied multimodal language model[J]. arXiv，2303.03378，2023.
[70] BROHAN A，BROWN N，CARBAJAL J，et al. RT-2:Vision-language-action models transfer web knowledge to robotic control[J]. arXiv，2307.15818，2023.
[71] SHIRAI K，BELTRAN-HERNANDEZ C C，HAMAYA M，et al. Vision-language interpreter for robot task planning[C]//2024 IEEE International Conference on Robotics and Automation (ICRA). 2024:2051-2058.
[72] XIE J，LIU S，WANG X. Framework for a closed-loop cooperative human cyber-physical system for the mining industry driven by VR and AR:MHCPS[J]. Computers & Industrial Engineering，2022，168:108050.
[73] WANG Z，BAI X，ZHANG S，et al. M-AR:A visual representation of manual operation precision in AR assembly[J]. International Journal of Human-Computer Interaction，2021，37(19):1799-1814.
[74] FANG W，FAN W，JI W，et al. Distributed cognition based localization for AR-aided collaborative assembly in industrial environments[J]. Robotics and Computer- Integrated Manufacturing，2022，75:102292.
[75] XIE J，LIU Y，WANG X，et al. A new XR-based human- robot collaboration assembly system based on industrial metaverse[J]. Journal of Manufacturing Systems，2024，74:949-964.
[76] VANNESTE P，DEKEYSER K，ULLAURI L A P，et al. Towards tailored cognitive support in augmented reality assembly work instructions[J]. Journal of Computer Assisted Learning，2024，40(2):797-811.
[77] FOGLI D，GARGIONI L，GUIDA G，et al. A hybrid approach to user-oriented programming of collaborative robots[J]. Robotics and Computer-Integrated Manufacturing，2022，73:102234.
[78] KROEMER O，NIEKUM S，KONIDARIS G. A review of robot learning for manipulation:Challenges，representations，and algorithms[J]. Journal of Machine Learning Research，2021，22(30):1-82.
[79] SCHAAL S. Learning from demonstration[C]//Advances in Neural Information Processing Systems，1996，9:1-6.
[80] SCHOU C，ANDERSEN R S，CHRYSOSTOMOU D，et al. Skill-based instruction of collaborative robots in industrial settings[J]. Robotics and Computer-Integrated Manufacturing，2018，53:72-80.
[81] YIN Y，ZHENG P，LI C，et al. Enhancing human-guided robotic assembly:AR-assisted DT for skill-based and low- code programming[J]. Journal of Manufacturing Systems，2024，74:676-689.
[82] ZHAO T Z，KUMAR V，LEVINE S，et al. Learning fine-grained bimanual manipulation with low-cost hardware[EB/OL]. (2023-04-23)[2024-10-09]. https://arxiv.org/abs/2304.13705v1.
[83] ZENG C，YANG C，CHENG H，et al. Simultaneously encoding movement and sEMG-based stiffness for robotic skill learning[J]. IEEE Transactions on Industrial Informatics，2021，17(2):1244-1252.
[84] WU H，YAN W，XU Z，et al. A framework of robot skill learning from complex and long-horizon tasks[J]. IEEE Transactions on Automation Science and Engineering，2022，19(4):3628-3638.
[85] WANG W，LI R，CHEN Y，et al. Facilitating human- robot collaborative tasks by teaching-learning-collaboration from human demonstrations[J]. IEEE Transactions on Automation Science and Engineering，2019，16(2):640-653.
[86] HU Z，ZHENG Y，PAN J. Grasping living objects with adversarial behaviors using inverse reinforcement learning[J]. IEEE Transactions on Robotics，2023，39(2):1151-1163.
[87] APOLINARSKA A A，PACHER M，LI H，et al. Robotic assembly of timber joints using reinforcement learning[J]. Automation in Construction，2021，125:103569.
[88] SUN H，ZHANG T，HAN J，et al. A fast transfer reinforcement learning model for transferring force-based human speed adjustment skills to robots for collaborative assembly posture alignment[J]. Advanced Engineering Informatics，2024，62:102836.
[89] GUPTA A，SAVARESE S，GANGULI S，et al. Embodied intelligence via learning and evolution[J]. Nature Communications，2021，12(1):5721.
[90] GUZMAN L，MORELLAS V，PAPANIKOLOPOULOS N. Robotic embodiment of human-like motor skills via reinforcement learning[J]. IEEE Robotics and Automation Letters，2022，7(2):3711-3717.
[91] KOMULAINEN T M，SANNERUD A R. Learning transfer through industrial simulator training:Petroleum industry case[J]. Cogent Education，2018，5(1):1554790.
[92] SHAYESTEH S，OJHA A，LIU Y，et al. Human-robot teaming in construction:Evaluative safety training through the integration of immersive technologies and wearable physiological sensing[J]. Safety Science，2023，159:106019.
[93] ESTRADA J，PAHEDING S，YANG X，et al. Deep- learning-incorporated augmented reality application for engineering lab training[J]. APPLIED SCIENCES- BASEL，2022，12(10):5159.
[94] EHRLINGER L，WÖSS W. Towards a definition of knowledge graphs[C]//International Conference on Semantic Systems. 2016:1-6.
[95] 黄恒琪，于娟，廖晓，等. 知识图谱研究综述[J]. 计算机系统应用，2019，28(6):1-12. HUANG Hengqi，YU Juan，LIAO Xiao，et al. Review on knowledge graphs[J]. Computer Systems Applications，2019，28(6):1-12.
[96] WANG G，HE J. A Bibliometric analysis of recent developments and trends in knowledge graph research (2013-2022)[J]. IEEE Access，2024，12:32005-32013.
[97] GUO H，LIU K. Hierarchical ontology graph for solving semantic issues in decision support systems[C]// Proceedings of the 21st International Conference on Enterprise Information Systems (ICEIS)，Setubal:Scitepress，2019:483-487.
[98] ZHAO G，ZHANG X. Domain-specific ontology concept extraction and hierarchy extension[C]//Proceedings of the 2nd International Conference on Natural Language Processing and Information Retrieval. New York，NY，USA:Association for Computing Machinery，2018:60-64.
[99] YUAN J，LI H. Research on the standardization model of data semantics in the knowledge graph construction of oil&gas industry[J]. Computer Standards & Interfaces，2023，84:103705.
[100] CHEN Y，FU L，QIU J，et al. Semantic based logistics data exchange model[C]//2022 7th International Conference on Intelligent Computing and Signal Processing (ICSP). 2022:928-933.
[101] KRÖTZSCH M，THOST V. Ontologies for knowledge graphs:Breaking the rules[C]//The Semantic Web-ISWC 2016:15th International Semantic Web Conference，Kobe，Japan，October 17-21，2016，Proceedings，Part I 15. Springer International Publishing，2016:376-392.
[102] KUMAR N，KUMAR M，SINGH M. Automated ontology generation from a plain text using statistical and NLP techniques[J]. International Journal of System Assurance Engineering and Management，2016，7(S1):282-293.
[103] KUWANA A，OBA A，SAWAI R，et al. Automatic taxonomy classification by pretrained language model[J]. Electronics，2021，10(21):2656.
[104] XU D，CHEN W，PENG W，et al. Large language models for generative information extraction:A survey[J]. Frontiers of Computer Science，2024，18(6):186357.
[105] QIAO B，ZOU Z，HUANG Y，et al. A joint model for entity and relation extraction based on BERT[J]. Neural Computing and Applications，2022，34(5):3471-3481.
[106] DAGDELEN J，DUNN A，LEE S，et al. Structured information extraction from scientific text with large language models[J]. Nature Communications，2024，15(1):1418.
[107] ZAFARIAN A，ROKNI A，KHADIVI S，et al. Semi-supervised learning for named entity recognition using weakly labeled training data[C]//2015 The International Symposium on Artificial Intelligence and Signal Processing (AISP). 2015:129-135.
[108] XU T，GUO C，DU L，et al. A method for traditional chinese medicine knowledge graph dynamic construction[C]// Proceedings of the 5th International Conference on Big Data Technologies. New York，NY，USA:Association for Computing Machinery，2022:196-202.
[109] PAN Z，SU C，DENG Y，et al. Video2Entities:A computer vision-based entity extraction framework for updating the architecture，engineering and construction industry knowledge graphs[J]. Automation in Construction，2021，125:103617.
[110] ZHENG C，FENG J，FU Z，et al. Multimodal relation extraction with efficient graph alignment[C]// Proceedings of the 29th ACM International Conference on Multimedia. New York，NY，USA:Association for Computing Machinery，2021:5298-5306.
[111] BORDES A，USUNIER N，GARCIA-DURAN A，et al. Translating embeddings for modeling multi-relational data[C]//Advances in Neural Information Processing Systems:Vol. 26. Curran Associates，Inc.，2013.
[112] ASMARA S M，SAHABUDIN N A，NADIAH ISMAIL N S，et al. A review of knowledge graph embedding methods of transe，transh and transr for missing links[C]// 2023 IEEE 8th International Conference On Software Engineering and Computer Systems (ICSECS). 2023:470-475.
[113] JIA Y，WANG Y，JIN X，et al. Knowledge graph embedding:A locally and temporally adaptive translation- based approach[J]. ACM Trans. Web，2017，12(2):8:1-8:33.
[114] LI Y，ZHU C. TransE-MTP:A new representation learning method for knowledge graph embedding with multi-translation principles and transe[J]. Electronics，2024，13(16):3171.
[115] SUN Z，DENG Z H，NIE J Y，et al. RotatE:Knowledge graph embedding by relational rotation in complex space[EB/OL]. (2019-02-26)[2024-08-28]. https://arxiv.org/abs/1902.10197v1.
[116] DETTMERS T，MINERVINI P，STENETORP P，et al. Convolutional 2D knowledge graph embeddings[J]. Proceedings of the AAAI Conference on Artificial Intelligence，2018，32(1):1811-1818.
[117] ZHAO X，ZHANG J，CAO Y，et al. Spatio-temporal knowledge embedding method considering the lifecycle of geographical entities[J]. International Journal of Applied Earth Observation and Geoinformation，2024，131:103967.
[118] YANG B，YIH W，HE X，et al. Embedding entities and relations for learning and inference in knowledge bases[EB/OL]. (2014-12-20)[2024-08-28]. https://arxiv.org/abs/1412.6575v4.
[119] TROUILLON T，WELBL J，RIEDEL S，et al. Complex embeddings for simple link prediction[C]//Proceedings of The 33rd International Conference on Machine Learning. PMLR，2016:2071-2080.
[120] MA Y，TRESP V，DAXBERGER E A. Embedding models for episodic knowledge graphs[J]. Journal of Web Semantics，2019，59:100490.
[121] WANG J，ZHANG W，CHEN X，et al. 3DRTE:3D rotation embedding in temporal knowledge graph[J]. IEEE Access，2020，8:207515-207523.
[122] XU X，LIN K，YANG Y，et al. Joint Feature Synthesis and embedding:Adversarial cross-modal retrieval revisited[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence，2022，44(6):3030-3047.
[123] LI Y，SU H，QI C R，et al. Joint embeddings of shapes and images via CNN image purification[J]. ACM Trans. Graph.，2015，34(6):234:1-234:12.
[124] MITHUN N C，LI J，METZE F，et al. Joint embeddings with multimodal cues for video-text retrieval[J]. International Journal of Multimedia Information Retrieval，2019，8(1):3-18.
[125] NAWAZ S，CALEFATI A，JANJUA M K，et al. Learning fused representations for large-scale multimodal classification[J]. IEEE Sensors Letters，2019，3(1):1-4.
[126] HUANG F，ZHANG X，XU J，et al. Network embedding by fusing multimodal contents and links[J]. Knowledge- Based Systems，2019，171:44-55.
[127] HAZARIKA D，PORIA S，ZIMMERMANN R，et al. Conversational transfer learning for emotion recognition[J]. Information Fusion，2021，65:1-12.
[128] YUAN Y，QU A. High-order joint embedding for multi-level link prediction[J]. Journal of the American Statistical Association，2023，118(543):1692-1706.
[129] GAO J，LYU T，XIONG F，et al. Predicting the survival of cancer patients with multimodal graph neural network[J]. IEEE/ACM Transactions on Computational Biology and Bioinformatics，2022，19(2):699-709.
[130] TAO Z，WEI Y，WANG X，et al. MGAT:multimodal graph attention network for recommendation[J]. Information Processing & Management，2020，57(5):102277.
[131] ZENG Z，CHENG Q，SI Y. Logical rule-based knowledge graph reasoning:A comprehensive survey[J]. Mathematics，2023，11(21):4486.
[132] 田玲，张谨川，张晋豪，等. 知识图谱综述——表示、构建、推理与知识超图理论[J]. 计算机应用，2021，41(8):2161-2186. TIAN Ling，ZHANG Jinchuan，ZHANG Jinhao，et al. Knowledge graph survey:Representation，construction，reasoning and knowledge hypergraph theory[J]. Journal of Computer Applications，2021，41(8):2161-2186.
[133] MAYR M，AHMAD F，CHATZILYGEROUDIS K，et al. Combining planning，reasoning and reinforcement learning to solve industrial robot tasks[J]. arXiv，2212.03570，2022.
[134] THIKE P H，XU Z，CHENG Y，et al. Materials failure analysis utilizing rule-case based hybrid reasoning method[J]. Engineering Failure Analysis，2019，95:300-311.
[135] SHI X，TIAN X，GU J，et al. A hybrid approach of case- and rule-based reasoning to assembly sequence planning[J]. The International Journal of Advanced Manufacturing Technology，2023，127(1):221-236.
[136] SMITH F，PROIETTI M. Rule-based behavioral reasoning on semantic business processes[C]//ICAART (2). 2013:130-143.
[137] ZHANG L，YU M，GAO T，et al. MCMH:Learning multi-chain multi-hop rules for knowledge graph reasoning[EB/OL]. (2020-10-05)[2024-08-28]. https://arxiv.org/abs/2010.01735v1.
[138] WANG J，HUANG Y. Logical reasoning over knowledge graphs in vector space[C]//Second International Symposium on Computer Applications and Information Systems (ISCAIS 2023):Vol. 12721. SPIE，2023:272-278.
[139] TANG X，ZHU S，LIANG Y，et al. RulE:Knowledge graph reasoning with rule embedding[C]//Findings of the Association for Computational Linguistics ACL 2024. 2024:4316-4335.
[140] NICKEL M，MURPHY K，TRESP V，et al. A review of relational machine learning for knowledge graphs[J]. Proceedings of the IEEE，2016，104(1):11-33.
[141] MA T，LV S，HUANG L，et al. HiAM:A hierarchical attention based model for knowledge graph multi-hop reasoning[J]. Neural Networks，2021，143:261-270.
[142] ZHANG Y，JIN L，LI X，et al. Edge-aware graph neural network for multi-hop path reasoning over knowledge base[J]. Computational Intelligence and Neuroscience，2022，2022(1):4734179.
[143] TRIVEDI R，DAI H，WANG Y，et al. Know-evolve: Deep temporal reasoning for dynamic knowledge graphs[C]//Proceedings of the 34th International Conference on Machine Learning. PMLR，2017:3462-3471.
[144] GUO T，YANG Q，WANG C，et al. Knowledge navigator:Leveraging large language models for enhanced reasoning over knowledge graph[J]. Complex & Intelligent Systems，2024，10(5):7063-7076.
[145] ABU-RASHEED H，WEBER C，FATHI M. Knowledge graphs as context sources for llm-based explanations of learning recommendations[J]. arXiv，2403.03008，2024.
[146] LIU H，ZHOU S，CHEN C，et al. Dynamic knowledge graph reasoning based on deep reinforcement learning[J]. Knowledge-Based Systems，2022，241:108235.
[147] BAI L，YU W，CHEN M，et al. Multi-hop reasoning over paths in temporal knowledge graphs using reinforcement learning[J]. Applied Soft Computing，2021，103:107144.
[148] KIM Y J，KWAK B，KIM Y，et al. Modularized transfer learning with multiple knowledge graphs for zero-shot commonsense reasoning[J]. arXiv，2206. 03715.
[149] WU Y，ZHOU J. CogTrans:A cognitive transfer learning-based self-attention mechanism architecture for knowledge graph reasoning[J]. Proceedings of the Annual Meeting of the Cognitive Science Society，2023，45(45):1893-1898.