Abstract: An artificial intelligence architecture—the chain-of-thought (CoT) paradigm text-based multimodal intelligent agent—for operation and maintenance (O&M) of mechanical equipment is proposed. Firstly, to address the challenge of constructing high-quality, large-scale monitoring data-to-fault mode mapping datasets in real-world engineering applications, a chain-of-thought dataset construction strategy integrating monitoring signals, mathematical features, text descriptions, and fault mode is proposed. Based on this, a signal-to-text (Sig2Txt) model driven by a signal-text data generator is developed. Subsequently, a high-quality specialized textual dataset for O&M in the mechanical equipment domain is created, and an intelligent O&M-specialized large language model is established through instruction fine-tuning on a general large language model. Finally, by organically integrating the above models based on large model intelligent agent technology and guided by the operational thinking patterns of human experts in equipment maintenance, a chain-of-thought paradigm text-based multimodal intelligent agent for intelligent O&M is formed. Testing results indicate that this model can achieve chain-of-thought parsing and mapping from multimodal input decision-making, with an accuracy exceeding 70% on ISO Category III vibration analyst test questions, thus reaching expert-level performance. In evaluations with engineering cases and publicly available multimodal datasets, the proposed model outperforms existing mainstream large models. More importantly, owing to the proposed low-cost, high-quality multimodal CoT large-scale dataset construction framework, and the unique advantages of a “text-based” approach in terms of encompassing knowledge, high-level abstraction, and interpretability, the model shows considerable scalability and development potential.

Key words: intelligent operation and maintenance, multimodal model, large language model, fault diagnosis, intelligent agent