A Human-cyber-physical Data Fusion Knowledge Graph Construction Method for Equipment Maintenance

doi:10.3901/JME.2025.17.215

Abstract

Abstract: Equipment maintenance is an important basis to ensure normal production, the existing intelligent maintenance technologies mainly rely on signal analysis, data mining or expert knowledge reuse. However, with the improvement of the automation and integration degree of production equipment, the relationships among the characteristic signals of various operating anomalies, multi-source causes and maintenance schemes present higher fuzziness and complexity, the integration analysis of signals, data and knowledge is the key to improve the accuracy and efficiency of equipment maintenance. Therefore, knowledge graph technology is used to integrate the ternary data of “human”, “cyber” and “physical” to support the abnormal diagnosis and maintenance scheme decision of complex equipment, improve the intelligent degree of equipment maintenance, and avoid the one-sidedness of decision. Firstly, the ternary man-machine object data in the field of equipment maintenance is defined and the ternary ontology design is completed to guide the construction of knowledge graph data layer. Secondly, preprocessing is conducted on the ternary data of human-cyber-physical and a unified joint entity and relation extraction model with mixed attention，MAREL is built to automatically extract knowledge from the ternary data of human-cyber-physical, and to establish associative relationships between them, thereby achieving the fusion of ternary human-cyber-physical data; MAREL dissolves the task into two related decoding modules to solve the entity overlap problem, and the mixed attention mechanism is used to enhance the long text processing capability of the model, the test on the Chinese data set SKE proves that MAREL has excellent performance. Finally, the construction of human-cyber-physical knowledge graph for the maintenance of robot equipment in an automobile production workshop is taken as an example, the effectiveness of the proposed method is verified, results show that the knowledge graph can effectively integrate the ternary data of “human”, “cyber” and “physical”, and provide decision support for intelligent equipment maintenance.

Key words: equipment maintenance, human-cyber-physical, knowledge graph, data fusion, ontology, joint extraction

CLC Number:

TP319
TH17

YANG Bo, SHEN Xiaoyu, WANG Shilong, HE Yan, DU Kaze. A Human-cyber-physical Data Fusion Knowledge Graph Construction Method for Equipment Maintenance[J]. Journal of Mechanical Engineering, 2025, 61(17): 215-232.

References

[1] CARVALHO T P, SOARES F A A M N, VITA R, et al. A systematic literature review of machine learning methods applied to predictive maintenance[J]. Computers & Industrial Engineering, 2019, 137: 106024.
[2] 沈君贤, 马天池, 宋狄, 等. 基于可解释选择性集成框架的离心风机叶片裂纹损伤检测[J]. 机械工程学报, 2024, 60(12): 183-193. SHEN Junxian, MA Tianchi, SONG Di, et al. Crack damage detection of centrifugal fan blades based on interpretable ensemble selection framework[J]. Journal of Mechanical Engineering, 2024, 60(12): 183-193.
[3] 韩特, 李彦夫, 雷亚国, 等. 融合图标签传播和判别特征增强的工业机器人关键部件半监督故障诊断方法[J]. 机械工程学报, 2022, 58(17): 116-124. HAN Te, LI Yanfu, LEI Yaguo, et al. Semi-supervised fault diagnosis method via graph label propagation and discriminative feature enhancement for critical components of industrial robot[J]. Journal of Mechanical Engineering, 2022, 58(17): 116-124.
[4] TABEMIK D, SELA S, SKVARC J, et al. Segmentation-based deep-learning approach for surface-defect detection[J]. Journal of Intelligent Manufacturing, 2020, 31(3): 759-776.
[5] 张显程, 谷行行, 刘宇, 等. 基于工程损伤理论的高温装备可靠性评估与运维管理[J]. 机械工程学报, 2024, 60(13): 154-172. ZHANG Xiancheng, GU Hanghang, LIU Yu, et al. Engineering damage theory-based reliability assessment and management of high-temperature equipment[J]. Journal of Mechanical Engineering, 2024, 60(13): 154-172.
[6] 黄包裕, 张永祥, 赵磊. 基于布谷鸟搜索算法和最大二阶循环平稳盲解卷积的滚动轴承故障诊断方法[J]. 机械工程学报, 2021, 57(9): 99-107. HUANG Baoyu, ZAHNG Yongxiang, ZHAO Lei. Research on fault diagnosis method of rolling bearings based on cuckoo search algorithm and maximum second order cyclostationary blind deconvolution[J]. Journal of Mechanical Engineering, 2021, 57(9): 99-107.
[7] LUO W, HU T, YE Y, et al. A hybrid predictive maintenance approach for CNC machine tool driven by digital twin[J]. Robotics and Computer-Integrated Manufacturing, 2020, 65: 101974.
[8] WU C, JIANG P, DING C, et al. Intelligent fault diagnosis of rotating machinery based on one-dimensional convolutional neural network[J]. Computers in Industry, 2019, 108: 53-61.
[9] 马波, 高金吉, 江志农. 基于专家思维的多维度故障诊断方法[J]. 机械工程学报, 2017, 53(23): 32-38. MA Bo, GAO Jinji, JIANG Zhinong. Multi-dimensional fault diagnosis method based on expert thinking[J]. Journal of Mechanical Engineering, 2017, 53(23): 32-38.
[10] HU M, LIU Y. E-maintenance platform design for public infrastructure maintenance based on IFC ontology and Semantic Web services[J]. Concurrency and Computation: Practice and Experience, 2020, 32(6): e5204.
[11] 万庆祝, 赵蕾. 基于暂态特征的配电网线路故障定位专家系统研究[J]. 电气工程学报, 2016, 11(11): 52-57. WAN Qingzhu, ZHAO Lei. Research on fault location expert system of distribution network based on transient characteristics[J]. Journal of Electrical Engineering, 2016, 11(11): 52-57.
[12] 董津, 王坚, 王兆平. 面向制造领域人机物三元数据融合的本体自动化构建方法[J]. 控制与决策, 2022, 37(5): 1251-1257. DONG Jin, WANG Jian, WANG Zhaoping. Automatic ontology construction for human-cyber-physical data fusion in manufacturing domain[J]. Control and Decision, 2022, 37(5): 1251-1257.
[13] PUJARA J, MIAO H, GETOOR L, et al. Ontology-Aware partitioning for knowledge graph identification[C]//Workshop on Automated Knowledge Base Construction. New York: Assoc Computing Machinery, 2013, 19-24.
[14] LI W, PENG R, WANG Y, et al. Knowledge graph based natural language generation with adapted pointer-generator networks[J]. Neurocomputing, 2020, 382: 174-187.
[15] HOLZINGER A, MALLE B, SARANTI A, et al. Towards multi-modal causability with graph neural networks enabling information fusion for explainable AI[J]. Information Fusion, 2021, 71: 28-37.
[16] LI Q, YANG B, WANG S, et al. A fine-grained flexible graph convolution network for visual inspection of resistance spot welds using cross-domain features[J]. Journal of Manufacturing Processes, 2022, 78: 319-329.
[17] YI Z, WANG X, OUNIS I, et al. Multi-modal graph contrastive learning for micro-video recommendation[C]//Research and Development in Information Retrieval. New York: Assoc Computing Machinery, 2022, 1807-1811.
[18] CHEN X, JIA S, XIANG Y. A review: knowledge reasoning over knowledge graph[J]. Expert Systems with Applications, 2020, 141: 112948.
[19] NUNEZ D L, BORSATO M. OntoProg: An ontology-based model for implementing prognostics health management in mechanical machines[J]. Advanced Engineering Informatics, 2018, 38: 746-759.
[20] ZHOU A, YU D, ZHANG W. A research on intelligent fault diagnosis of wind turbines based on ontology and FMECA[J]. Advanced Engineering Informatics, 2015, 29(1): 115-125.
[21] DONG C, ZHANG J, ZONG C, et al. Character-Based lstm-crf with radical-level features for chinese named entity recognition[C]//Natural Language Processing and Chinese Computing/Computer Processing of Oriental Languages. Berlin, Germany: Springer-Verlagberlin, 2016, 10102: 239-250.
[22] SOARES L B, FITZGERALD N, LING J, et al. Matching the blanks: distributional similarity for relation learning[C]//Association for Computational Linguistics. Stroudsburg: Assoc Computational Linguistics-Acl, 2019, 2895-2905.
[23] REN X, WU Z, HE W, et al. CoType: joint extraction of typed entities and relations with knowledge bases[C]//International Conference on World Wide Web. Republic and Canton of Geneva, Switzerland: International World Wide Web Conferences Steering Committee, 2017, 1015-1024.
[24] ZHENG S, XU J, ZHOU P, et al. A neural network framework for relation extraction: learning entity semantic and relation pattern[J]. Knowledge-Based Systems, 2016, 114: 12-23.
[25] WANG P, YANG L T, LI J, et al. Data fusion in cyber-physical-social systems: State-of-the-art and perspectives[J]. Information Fusion, 2019, 51: 42-57.
[26] ALI F, EL-SAPPAGHS, ISLAMS M R, et al. An intelligent healthcare monitoring framework using wearable sensors and social networking data[J]. Future Generation Computer Systems-the International Journal of Escience, 2021, 114: 23-43.
[27] mirrors/zhengyima/kg-baseline-pytorch[EB/OL]. GitCode./2022-12-08. https://gitcode.net/mirrors/zhengyima/kg-baseline-pytorch?utm_source=csdn_github_accelerator.
[28] WEI Z, SU J, WANG Y, et al. A novel cascade binary tagging framework for relational triple extraction[C]//Association-for-Computational-Linguistics. Stroudsburg: Assoc Computational Linguistics-Acl, 2020, 1476-1488.
[29] An Overview of the 2019 Language and Intelligence Challenge[EB/OL]./2022-11-21. https://www.zhangqiaokeyan.com/academic-conference-foreign_ccf-internatial-cference-natural-language-processi_thesis/020512714508.html.
[30] FU T J, LI P H, MA W Y. GraphRel: modeling text as relational graphs for joint entity and relation extraction[C]//Association for Computational Linguistics. Stroudsburg: Assoc Computational Linguistics-Acl, 2019, 1409-1418.
[31] ZENG X, ZENG D, HE S, et al. Extracting relational facts by an end-to-end neural model with copy mechanism[C]//Association for Computational Linguistics. Stroudsburg: Assoc Computational Linguistics-Acl, 2018, 506-514.
[32] ZHENG S, WANG F, BAO H, et al. Joint extraction of entities and relations based on a novel tagging scheme[C]//Association for Computational Linguistics. Stroudsburg: Assoc Computational Linguistics-Acl, 2017, 1227-1236.
[33] 陈仁杰, 郑小盈, 祝永新. 融合实体类别信息的实体关系联合抽取[J]. 计算机工程, 2022, 48(3): 46-53. CHEN Renjie, ZHENG Xiaoying, ZHU Yongxin. Joint entity and relation extraction fusing entity type information[J]. Computer Engineering, 2022, 48(3): 46-53.