Research on Deep Reinforcement Learning-based Intelligent Car-following Control and Energy Management Strategy for Hybrid Electric Vehicles

doi:10.3901/JME.2021.22.237

Abstract

Abstract: Limited by the cost, high-speed on-off valves are often used in the electro-hydraulic braking system. However, the knocking noise of the solenoid valve, hydraulic impact noise and pressure fluctuation in the switch control of the hydraulic valve lead to the quality and precision of brake control to deteriorate. Therefore, the key to the design of the high-speed on-off valve is to realize the linearization by pulse width modulation (PWM) control. The dynamic characteristics of high-speed on-off valve are affected by transient flow force, nonlinear electromagnetic force, mechanical inertia and spring force. Therefore, the dynamic model and joint simulation model of high-speed on-off valve and hydraulic control unit (HCU) are established. Through simulation and experimental verification, the influence of structural parameters such as poppet angle, orifice diameter, air gap size on the linear characteristics of solenoid valve is analyzed. The reasonable structure parameters of the solenoid valve are designed and applied to a hydraulic control circuit to widen the linear working range and meet the requirements of automobile braking safety control.

Key words: hybrid electric vehicle, deep reinforcement learning, car-following control, energy management

CLC Number:

U461

TANG Xiaolin, CHEN Jiaxin, LIU Teng, LI Jiacheng, HU Xiaosong. Research on Deep Reinforcement Learning-based Intelligent Car-following Control and Energy Management Strategy for Hybrid Electric Vehicles[J]. Journal of Mechanical Engineering, 2021, 57(22): 237-246.

References

[1] 李克强, 戴一凡, 李升波, 等. 智能网联汽车(ICV)技术的发展现状及趋势[J]. 汽车安全与节能学报, 2017, 8(1):1-14. LI Keqiang, DAI Yifan, LI Shengbo, et al. State-of-the-art and technical trends of intelligent and connected vehicles[J]. Journal of Automotive Safety and Energy, 2017, 8(1):1-14.
[2] 欧阳明高. 中国新能源汽车的研发及展望[J]. 科技导报, 2016, 34(6):13-20. OUYANG Minggao. New energy vehicle research and development in China[J]. Science and Technology Review, 2016, 34(6):13-20.
[3] TRAN D D, VAFAEIPOUR M, EL BAGHDADI M, et al. Thorough state-of-the-art analysis of electric and hybrid vehicle powertrains:Topologies and integrated energy management strategies[J]. Renew. Sust. Energ. Rev., 2020, 119.
[4] 张风奇, 胡晓松, 许康辉, 等. 混合动力汽车模型预测能量管理研究现状与展望[J]. 机械工程学报, 2019, 55(10):86-108. ZHANG Fengqi, HU Xiaosong, XU Kanghui, et al. Current status and prospects for model predictive energy management in hybrid electric vehicles[J]. Journal of Mechanical Engineering, 2019, 55(10):86-108.
[5] 唐小林, 李珊珊, 王红, 等. 网联环境下基于分层式模型预测控制的车队能量控制策略研究[J]. 机械工程学报, 2020, 56(14):119-128. TANG Xiaolin, LI Shanshan, WANG Hong, et al. Research on energy control strategy based on hierarchical model predictive control in connected environment[J]. Journal of Mechanical Engineering, 2020, 56(14):119-128.
[6] SILVER D, HUANG A, MADDISDN C, et al. Mastering the game of Go with deep neural networks and tree search[J]. Nature, 2016, 529(7587):484-489.
[7] SILVER D, SCHRITTWIESER J, SIMONYAN K, et al. Mastering the game of Go without human knowledge[J]. Nature, 2017, 550(7676):354-359.
[8] LIU T, HU X S, LI S B, et al. Reinforcement learning optimized look-ahead energy management of a parallel hybrid electric vehicle[J]. IEEE-ASME T. Mech. 2017, 22(4):1497-1507.
[9] TAN H C, ZHANG H L, PENG J K, et al. Energy management of hybrid electric bus based on deep reinforcement learning in continuous state and action space[J]. Energ. Convers. Manage., 2019(195):548-560.
[10] VOLODYMYR M, KORAY K, DAVID S, et al. Human-level control through deep reinforcement learning[J]. Nature, 2015, 518(7540):529-533.
[11] WU J D, HE H W, PENG J K. Continuous reinforcement learning of energy management with deep Q network for a power split hybrid electric bus[J]. Appl. Energ., 2018(222):799-811.
[12] LI Y C, HE H W, PENG J K, et al. Energy management strategy for a series hybrid electric vehicle using improved deep Q-network learning algorithm with prioritized replay[C/CD]//DEStech Transactions on Environment, Energy and Earth Sciences, 2018.
[13] HAN X F, HE H W, WU J D. Energy management based on reinforcement learning with double deep Q-learning for a hybrid electric tracked vehicle[J]. Appl. Energ., 2019(254):113708.
[14] 赵星宇, 丁世飞. 深度强化学习研究综述[J]. 计算机科学, 2018, 45(7):1-6. ZHAO Xingyu, DING Shifei. Research on deep reinforcement learning[J]. Computer Science, 2018, 45(7):1-6.
[15] 刘腾. 混合动力车辆强化学习能量管理研究[D]. 北京:北京理工大学, 2017. LIU Teng. Reinforcement learning-based energy management for hybrid electric vehicles[D]. Beijing:Beijing Institute of Technology, 2017.
[16] 胡悦. 混合动力电动汽车控制系统设计与能量管理策略研究[D]. 北京:中国科学院大学(中国科学院深圳先进技术研究院), 2018. HU Yue. Research on control system design and energy management strategy of hybrid electric vehicle[D]. Beijing:University of Chinese Academy of Sciences(Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences), 2018.