Abstract: In recent years, data-driven fault diagnosis methods have been widely researched. A prerequisite of ensuring those methods' effectiveness is that the data for training diagnosis models and data to be tested should be collected from the same machine and the working environment. However, it is very difficult to satisfy this prerequisite in the actual diagnosis problem, and only the historical fault data collected from other same-type machines or different operating conditions are available for training the diagnosis model. The validity of conventional data-driven fault diagnosis methods for the actual diagnosis scenarios between datasets with potential discrepancy has rarely been discussed yet. The possible factors that dominate the generalization performance of fault diagnosis methods are first analyzed theoretically, and then multiple cross-dataset diagnosis tasks are organized. Based on them the empirical analysis about the generalization performance of several data-driven fault diagnosis methods is conducted. It is found that the distribution discrepancy between datasets is a major factor to influence the generalization performance. Meanwhile, the further fundamental reason for generalization performance declines is also explained from the perspective of signal characteristics under both model difference and operating condition difference. The discussion is conducive to inspiring the studies of data-driven diagnosis methods that can handle actual diagnosis scenarios.

Key words: fault diagnosis, data-driven, generalization performance, empirical analysis