基于人体特征点提取和多维时间序列分类的驾驶行为识别方法

doi:10.3901/JME.2025.15.233

机械工程学报 ›› 2025, Vol. 61 ›› Issue (15): 233-246.doi: 10.3901/JME.2025.15.233

• 人因与具身智能 • 上一篇

扫码分享

基于人体特征点提取和多维时间序列分类的驾驶行为识别方法

李朝¹, 赵霞², 赵继康¹, 付锐¹, 王畅¹

1. 长安大学汽车学院西安 710064;
2. 江苏大学汽车工程研究院镇江 212013

收稿日期:2024-09-17 修回日期:2025-02-09 发布日期:2025-09-28
作者简介:李朝，男，1996年出生，博士研究生。主要研究方向为驾驶人行为建模、智能驾驶、人机共驾。E-mail：2019322001@chd.edu.cn;赵霞，女，1994年出生，博士，讲师。主要研究方向为驾驶人行为建模与分析、智能车辆驾驶决策。E-mail：zhaoxia@ujs.edu.cn;赵继康，男，2000年出生，硕士研究生。主要研究方向为驾驶人行为建模。E-mail：zhaojikang@chd.edu.cn;付锐，女，1965年出生，博士，教授，博士研究生导师。主要研究方向为人机共驾、人-车-路系统安全。E-mail：furui@chd.edu.cn;王畅(通信作者)，男，1984年出生，博士，教授，博士研究生导师。主要研究方向为交通安全、智能驾驶、人机交互。E-mail：wangchang@chd.edu.cn
基金资助:
陕西省重点研发计划(2021LLRH-04-01-01); 江苏省自然科学青年基金(BK20240870); 长安大学中央高校基本科研业务费专项资金(300102224501)资助项目。

Driving Behavior Recognition Based on Human Feature Point Extraction and Multidimensional Time Series Classification

LI Zhao¹, ZHAO Xia², ZHAO Jikang¹, FU Rui¹, WANG Chang¹

1. School of Automobile, Chang'an University, Xi'an 710064;
2. Automotive Engineering Research Institute, Jiangsu University, Zhenjiang 212013

Received:2024-09-17 Revised:2025-02-09 Published:2025-09-28

摘要/Abstract

摘要： 驾驶人行为识别对提高驾驶安全性和发展智能交通至关重要。由于不同驾驶环境和驾驶人特征存在较大差异，基于端到端深度学习的驾驶行为识别模型很难在不同数据集下保持较高的泛化性能。针对上述问题，提出基于人体特征点提取和多维时间序列分类的驾驶人行为识别方法。使用YOLOv8和人体姿态估计器蒸馏(Distillation for whole-body pose estimators,DWPose)跟踪驾驶人区域并提取驾驶人人体特征点矩阵。对特征点矩阵进行归一化、平滑和维度转换。分别建立了基于Transformer、Informer、时间卷积神经网络(Temporal convolutional network,TCN)和注意力机制-长短期记忆网络(Attention based long short-term memory,LSTM-Attention)的多维时间序列分类模型。结果表明，Informer模型的识别准确性最高，TCN模型的运行效率最高。当使用Driver-100-Day进行训练时，Informer在Driver-100-Day、Driver-100-Night和State Farm Driver 2数据集上的测试准确度分别为90.82%、88.77%和80.67%，相比于CNN-Transformer提高了24.56%、72.02%和67.57%。所提方法相比于基于单帧图像输入的模型在泛化性能方面有较大的改善，且能够达到较高的识别效率和准确度。

关键词: 智能驾驶, 驾驶行为识别, 人体特征点, 多维时间序列分类, 泛化性能

Abstract: Driver behavior recognition is crucial for both improving driving safety and developing intelligent transportation. Due to the large differences in different driving environments and driver characteristics, it is difficult for driving behavior recognition models based on end-to-end deep learning to maintain high generalization performance under different datasets. To address the above problems, a driver behavior recognition method based on human feature point extraction and multi-dimensional time series classification is proposed. YOLOv8 and distillation for whole-body pose estimators（DWPose） are used to track the driver region and extract the driver human feature point matrix. The feature point matrix is normalized, smoothed and dimensionally transformed. Multidimensional time series classification models based on Transformer, Informer, temporal convolutional neural network（TCN） and attention mechanism-long and short-term memory networks（LSTM-Attention） are established, respectively. The results show that the Informer model has the highest recognition accuracy and the TCN model has the highest operational efficiency. When trained with Driver-100-Day, Informer’s test accuracies on the Driver-100-Day, Driver-100-Night, and State Farm Driver 2 datasets are 90.82%, 88.77%, and 80.67%, respectively, which is higher than that of CNN-Transformer by 24.56%, 72.02% and 67.57%. The proposed method shows a major improvement in generalization compared to the model based on single frame image input and is able to arrive at higher recognition efficiency and accuracy.

Key words: intelligent driving, driving behavior recognition, human feature points, time series classification, generalization performance

中图分类号:

U471

李朝, 赵霞, 赵继康, 付锐, 王畅. 基于人体特征点提取和多维时间序列分类的驾驶行为识别方法[J]. 机械工程学报, 2025, 61(15): 233-246.

LI Zhao, ZHAO Xia, ZHAO Jikang, FU Rui, WANG Chang. Driving Behavior Recognition Based on Human Feature Point Extraction and Multidimensional Time Series Classification[J]. Journal of Mechanical Engineering, 2025, 61(15): 233-246.

参考文献

[1] KOAY H V,CHUAH J H,CHOW C O,et al. Detecting and recognizing driver distraction through various data modality using machine learning:A review, recent advances, simplified framework and open challenges (2014-2021)[J]. Engineering Applications of Artificial Intelligence,2022,115:105309.
[2] QU Y,HU H,LIU J,et al. Driver state monitoring technology for conditionally automated vehicles:Review and future prospects[J]. IEEE Transactions on Instrumentation and Measurement,2023,72:1-20.
[3] 张波,王文军,魏民国,等.基于机器视觉的驾驶人使用手持电话行为检测[J].吉林大学学报,2015,45(5):1688-1695.ZHANG Bo,WANG Wenjun, WEI Minguo, et al.Detection handheld phone use by driver based on machine vision[J]. Journal of Jilin University, 2015, 45(5):1688-1695.
[4] 程文冬,付锐,马勇,等.驾驶人在手机通话行为中的认知分心图像识别研究[J].中国公路学报,2021,34(5):168-181.CHENG Wendong,FU Rui,MA Yong,et al. Research on driver's cognitive distraction in mobile phone call behavior based on image recognition[J]. China Journal of Highway and Transport,2021,34(5):168-181.
[5] LI W,HUANG J,XIE G,et al. A survey on vision-based driver distraction analysis[J]. Journal of Systems Architecture,2021,121:102319.
[6] WANG J,CHAI W,VENKATACHALAPATHY A,et al.A survey on driver behavior analysis from in-vehicle cameras[J]. IEEE Transactions on Intelligent Transportation Systems,2022,23(8):10186-10209.
[7] XING Y, LÜC, WANG H, et al. Driver activity recognition for intelligent vehicles:A deep learning approach[J]. IEEE Transactions on Vehicular Technology,2019,68(6):5379-5390.
[8] 贺宜,鲁曼可,高嵩,等.基于Mobile ViT-CA模型的营运车辆驾驶人分心行为检测[J].中国公路学报,2024,37(1):194-204.HE Yi,LU Manke,GAO Song,et al,Distracted behavior detection of commercial vehicle drivers based on the mobilevit-ca model[J]. China Journal of Highway and Transport,2024,37(1):194-204.
[9] 柳长源,虎浩媛,毕晓君.双线性融合网络的驾驶员分心行为识别[J].北京邮电大学学报,2022,45(2):79-84.LIU Changyuan, HU Haoyuan, BI Xiaojun. Driver distraction recognition using bilinear fusion networks[J].Journal of Beijing University of Posts and Telecommunications,2022,45(2):79-84.
[10] LI B,CHEN J,HUANG Z,et al. A new unsupervised deep learning algorithm for fine-grained detection of driver distraction[J]. IEEE Transactions on Intelligent Transportation Systems,2022,23(10):19272-19284.
[11] 张斌,付俊怡,夏金祥.基于类间距优化的分心驾驶行为识别模型训练方法[J].汽车工程,2022,44(2):225-232.ZHANG Bin,FU Junyi,XIA Jinxiang. A metric space optimized method for driver distraction recognition model training[J]. Automotive Engineering, 2022, 44(2):225-232.
[12] LU M,HU Y,LU X. A pose-aware dynamic weighting model using feature integration for driver action recognition[J]. Engineering Applications of Artificial Intelligence,2022,113:104918.
[13] LIU D, YAMASAKI T, WANG Y, et al. Toward extremely lightweight distracted driver recognition with distillation-based neural architecture search and knowledge transfer[J]. IEEE Transactions on Intelligent Transportation Systems,2023,24(1):764-777.
[14] 曹立波,杨洒,艾昌硕,等.基于深度学习的分心驾驶行为检测方法[J].汽车技术,2023(6):49-54.CAO Libo,YANG Sa,AI Changshuo,et al. Distracted driving behavior detection based on deep learning[J].Automobile Technology,2023(6):49-54.
[15] JEGHAM I,ALOUANI I,BEN KHALIFA A,et al. Deep learning-based hard spatial attention for driver in-vehicle action monitoring[J]. Expert Systems with Applications,2023,219:119629.
[16] 尹智帅,钟恕,聂琳真,等.基于人体姿态估计的分心驾驶行为检测[J].中国公路学报,2022,35(6):312-323.YIN Zhishuai, ZHONG Shu, NIE Linzhen, et al.Distracted driving behavior detection based on human pose estimation[J]. China Journal of Highway and Transport,2022,35(6):312-323.
[17] RAMIS S,BUADES J M,PERALES F J,et al. A novel approach to cross dataset studies in facial expression recognition[J]. Multimedia Tools and Applications,2022,81(27):39507-39544.
[18] LIU Z,LI Y,YAO L,et al. Side-aware meta-learning for cross-dataset listener diagnosis with subjective tinnitus[J].IEEE Transactions on Neural Systems and Rehabilitation Engineering,2022,30:2352-2361.
[19] PRAJOD P,ANDRÉE J A E P. On the generalizability of ECG-based stress detection models[C]//202221st IEEE International Conference on Machine Learning and Applications (ICMLA). New York:IEEE,2022:549-554.
[20] WANG J,LI W,LI F,et al. 100-Driver:A large-scale,diverse dataset for distracted driver classification[J]. IEEE Transactions on Intelligent Transportation Systems,2023,24(7):7061-7072.
[21] ALOTAIBI M, ALOTAIBI B. Distracted driver classification using deep learning[J]. Signal,Image and Video Processing,2020,14(3):617-624.
[22] STACCHIO L,ANGELI A,LISANTI G,et al. Analyzing cultural relationships visual cues through deep learning models in a cross-dataset setting[J]. Neural Computing and Applications,2024:36(20):11727-11742.
[23] GARRUCHO L,KUSHIBAR K,JOUIDE S,et al.Domain generalization in deep learning based mass detection in mammography:A large-scale multi-center study[J]. Artificial Intelligence in Medicine,2022,132:102386.
[24] MOU L,CHANG J,ZHOU C,et al. Multimodal driver distraction detection using dual-channel network of CNN and transformer[J]. Expert Systems with Applications,2023,234:121066.
[25] BAHETI B, TALBAR S, GAJRE S. Towards computationally efficient and realtime distracted driver detection with mobilevgg network[J]. IEEE Transactions on Intelligent Vehicles,2020,5(4):565-574.

基于人体特征点提取和多维时间序列分类的驾驶行为识别方法

Driving Behavior Recognition Based on Human Feature Point Extraction and Multidimensional Time Series Classification

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 6

编辑推荐

Metrics

本文评价

[1]	孙念怡, 赵津, 黄磊, 王广玮. 面向隧道场景的智能车雷达与视觉协同感知研究[J]. 机械工程学报, 2025, 61(13): 80-95.
[2]	杨泽坤, 李韶华, 王振峰. 基于自适应变参数MPC的分布式驱动智能车轨迹跟踪控制[J]. 机械工程学报, 2024, 60(6): 363-377.
[3]	胡林, 杨冬兆, 张新, 章杰, 廖家才. 基于DQP-LMPC的智能车超车换道动态路径规划[J]. 机械工程学报, 2024, 60(10): 171-181.
[4]	李韶华, 杨泽坤, 王雪玮. 基于T-S模糊变权重MPC的智能车轨迹跟踪控制[J]. 机械工程学报, 2023, 59(4): 199-212.
[5]	彭湃, 耿可可, 殷国栋, 庄伟超, 刘帅鹏, 徐利伟. 基于传感器融合里程计的相机与激光雷达自动重标定方法[J]. 机械工程学报, 2021, 57(20): 206-214.
[6]	郑怀亮, 王日新, 杨远涛, 尹建程, 徐敏强. 数据驱动故障诊断方法泛化性能的经验性分析[J]. 机械工程学报, 2020, 56(9): 102-117.