Online Recognition Model Construction Method of Driver Emotion Based on Physiological Feature Mapping

doi:10.3901/JME.2022.20.379

Abstract

Abstract: High-precision driver’s emotion recognition model is one of the key problems for intelligent vehicle safety assisted driving systems. To solve this problem, a method is proposed to build driver’s emotion online recognition model through physiological feature mapping. The time-domain features of driver’s physiological signals of ECG and pulse under different emotions are extracted, the sample library of physiological signal and emotion correlation is established. An improved stacked autoencoder (I-SAE) neural network is proposed, an emotion model of physiological representation and off-line identification based on I-SAE is built. Both physiological signals and vehicle state data are collected for different emotions during the driving. The vehicle state characteristic parameter vector is made by principal component analysis and the physiological signal data is processed through I-SAE model. The driver’s emotion representation is transferred from physiology to vehicle state, and the correlation sample library of vehicle state and driver emotion are obtained. On this basis, a driver emotion online recognition model based on learning vector quantization (LVQ) is established. The experimental data show that the correct recognition rate of the driver's emotion online recognition model constructed by this method is more than 84%, which can meet the needs of intelligent systems such as vehicle safety distance early warning.

Key words: driver emotion, physiological signal, vehicle state, neural network

CLC Number:

TN915

WANG Yuefei, MA Weili, WANG Wenkang, WANG Chao, XIAO Kai. Online Recognition Model Construction Method of Driver Emotion Based on Physiological Feature Mapping[J]. Journal of Mechanical Engineering, 2022, 58(20): 379-390.

References

[1] 秦大同，詹森，曾育平，等. 基于驾驶风格识别的混合动力汽车能量管理策略[J]. 机械工程学报，2016，52(8):162-169. QIN Datong，ZHAN Sen，ZENG Yuping，et al. Hybrid electric vehicle energy management strategy based on driving style recognition[J]. Journal of Mechanical Engineering，2016，52(8):162-169.
[2] LIU Yaqi，WANG Xiaoyuan. Differences in driving intention transitions caused by driver’s emotion evolutions[J]. International Journal of Environmental Research and Public Health，2020，17(19):6962.
[3] SHOJAEILANGARI S，YAU W，NANDAKUMAR K，et al. Robust representation and recognition of facial emotions using extreme sparse learning[J]. IEEE Transactions on Image Processing，2015，24(7):2140-2152.
[4] ZHI Ruicong，ZHOU Caixia，LI Tingting，et al. Action unit analysis enhanced facial expression recognition by deep neural network evolution[J]. Neurocomputing，2021，42:135-148.
[5] 刘明周，蒋倩男，扈静. 基于面部几何特征及手部运动特征的驾驶员疲劳检测[J]. 机械工程学报，2019，55(2):18 -26. LIU Mingzhou，JIANG Qiannan，HU Jing. Driver fatigue detection based on facial geometric features and hand motion features[J]. Journal of Mechanical Engineering，2019，55(2):18-26.
[6] YANG Biao，CAO Jinmeng，NI Rongrong，et al. Facial expression recognition using weighted mixture deep neural network based on double-channel facial images[J]. IEEE Access，2018，6:4630-4640.
[7] RICHHARIYA B，GUPTA D. Facial expression recognition using iterative universum twin support vector machine[J]. Applied Soft Computing，2019，76:53-67.
[8] VICENTE F，HUANG Zehua，XIONG Xuehan，et al. Driver gaze tracking and eyes off the road detection system[J]. IEEE Transactions on Intelligent Transportation Systems，2015，16(4):2014-2027.
[9] 王跃飞，王文康，李洋，等. 基于分层网络模型的驾驶员消极驾驶状态在线感知方法:中国，109934171B[P]. 2020-03-17. WANG Yeifei，WANG Wenkang，Li Yang，et al. Hierarchical network model-based driver passive driving state online sensing method:China，109934171B[P]. 2020-03-17.
[10] HSU Y，WANG J，CHIANG W，et al. Automatic ECG-based emotion recognition in music listening[J]. IEEE Transactions on Affective Computing，2020，11(1):85-99.
[11] MINHAD K N，ALI S，REAZ M. Happy-anger emotions classifications from electrocardiogram signal for automobile driving safety and awareness[J]. Journal of Transport & Health，2017，7:75-89.
[12] ELISABETH R，ANTONIO J. Fractal dimension of EEG signals and heart dynamics in discrete emotional states[J]. Biological Psychology，2018，137:42-48.
[13] YANG C，FAHIER N，LI W，et al. A convolution neural network based emotion recognition system using multimodal physiological signals[C]// 2020 IEEE International Conference on Consumer Electronics - Taiwan (ICCE-Tai wan)，Taoyuan，Taiwan，IEEE，2020:1-2.
[14] DOMINGUEZ-JIMENEZ J A，CAMOP-LANDINES K C，MARTINEZ- SANTOS J C，et al. A machine learning model for emotion recognition from physiological signals[J]. Biomed Signal Process Control，2020，55:101646.
[15] LEE J，KIM J，SHIN M. Correlation analysis between Electrocardiography (ECG) and Photoplethysmogram (PPG) data for driver’s drowsiness detection using noise replacement method[J]. Procedia Computer Science，2017，116:421-426.
[16] WANG Xiaoyuan，GUO Yongqing，BAI Chenglin，et al. Driver’s intention identification with the involvement of emotional factors in two-lane roads[J]. IEEE Transactions on Intelligent Transportation Systems，2020:1-9.
[17] BOCKLISCH F，BOCKLISCH S，BEGGIATO M，et al. Adaptive fuzzy pattern classification for the online detection of driver lane change intention[J]. Neurocomputing，2017，262:148-158.
[18] DING Nan，MA Haoxuan，ZHAO Changuo，et al. Driver’s emotional state-based data anomaly detection for vehicular ad hoc networks[C]// 2019 IEEE International Conference on Smart Internet of Things (SmartIoT)，2019，Tianjin，China，IEEE，2019: 121-126.
[19] PENG Jinshuan，GUO Yingshi，FU Rui，et al. Multi-parameter prediction of drivers’ lane-changing behavior with neural network model[J]. Applied Ergonomics，2015，50:207-217.
[20] 陈鲲，茆志伟，张进杰，等. 基于和声搜索优化栈式自编码器的柴油发动机故障诊断[J]. 机械工程学报，2020，56(11):132-140. CHEN Kun，MAO Zhiwei，ZHANG Jinjie，et al. Diesel engine fault diagnosis based on acoustic search optimization stack autoencoder[J]. Journal of Mechanical Engineering，2020，56(11):132-140.
[21] MELIN P，AMEZCUA J，VALDEZ F，et al. A new neural network model based on the LVQ algorithm for multi-class classification of arrhythmias[J]. Information Sciences，2014，279:483-497.
[22] 刘白隽. 基于驾驶意图识别的HEV动力电池寿命在线管理研究[D]. 合肥:合肥工业大学，2019. LIU Baijun. Research on online management of HEV power battery life based on driving Intention recognition[D]. Hefei:Hefei University of Technology，2019.