Abstract: Driver behavior recognition is crucial for both improving driving safety and developing intelligent transportation. Due to the large differences in different driving environments and driver characteristics, it is difficult for driving behavior recognition models based on end-to-end deep learning to maintain high generalization performance under different datasets. To address the above problems, a driver behavior recognition method based on human feature point extraction and multi-dimensional time series classification is proposed. YOLOv8 and distillation for whole-body pose estimators（DWPose） are used to track the driver region and extract the driver human feature point matrix. The feature point matrix is normalized, smoothed and dimensionally transformed. Multidimensional time series classification models based on Transformer, Informer, temporal convolutional neural network（TCN） and attention mechanism-long and short-term memory networks（LSTM-Attention） are established, respectively. The results show that the Informer model has the highest recognition accuracy and the TCN model has the highest operational efficiency. When trained with Driver-100-Day, Informer’s test accuracies on the Driver-100-Day, Driver-100-Night, and State Farm Driver 2 datasets are 90.82%, 88.77%, and 80.67%, respectively, which is higher than that of CNN-Transformer by 24.56%, 72.02% and 67.57%. The proposed method shows a major improvement in generalization compared to the model based on single frame image input and is able to arrive at higher recognition efficiency and accuracy.

Key words: intelligent driving, driving behavior recognition, human feature points, time series classification, generalization performance