Research on Multi-objective EMS of Fuel Cell Hybrid Electric Vehicle under Car Following Scenario Based on Deep Reinforcement Learning

doi:10.3901/JME.2025.12.191

Abstract

Abstract: In order to improve the economy and safety of fuel cell hybrid electric vehicle(FCHEV), the energy management of hybrid power system output power distribution under car following scenario was studied. The research on the multi-objective energy management of FCHEV under car following scenario was divided into upper adaptive cruise control(ACC) strategy and lower energy management strategy(EMS). The proposed upper ACC strategy takes car following safety, driving comfort and economy into the scope of car following optimization constraints through deep deterministic policy gradient(DDPG), through controlling the acceleration of the rear vehicle to achieve car following of the front vehicle. The proposed lower level EMS strategy takes hydrogen consumption, state of charge(SOC) constraints, temperature of Lithium Ion Battery(LIB) and power system life control as optimization objectives, and realizes multi-objective power optimization allocation based on deep reinforcement learning. By coupling ACC and EMS, in which ACC provides EMS with the required power for vehicle movement, and EMS feeds back the hydrogen consumption, the economy of car following scenario can be further improved and the amount of calculation can be reduced. The simulation results show that under the proposed upper level car following control, the relative speed and following distance errors are 0.47 m∙s^-1 and 1.14 m, respectively, indicating good following performance; The proposed multi-objective EMS can reduce hydrogen consumption by 1.8% compared with the single objective strategy, and has a more stable SOC performance. The LIB temperature rise and life degradation are reduced by 63% and 83% respectively, and the comprehensive economic cost is reduced by 51%; Under car following scenario, compared with the traditional ACC, the proposed ACC-EMS framework further reduces the hydrogen consumption by 2.9% to ensure the safety, comfort and economy of FCHEV during car following process.

Key words: fuel cell hybrid electric vehicle, deep reinforcement learning, car following control, energy management

CLC Number:

U469

CAO Yanxu, WEI Zhongbao, SONG Ruoyang, HE Hongwen. Research on Multi-objective EMS of Fuel Cell Hybrid Electric Vehicle under Car Following Scenario Based on Deep Reinforcement Learning[J]. Journal of Mechanical Engineering, 2025, 61(12): 191-203.

References

[1] WEI Zhongbao，SONG Ruoyang，JI Dongxu，et al. Hierarchical thermal management for PEM fuel cell with machine learning approach[J]. Applied Thermal Engineering，2024，236：121544.
[2] ZHENG Chunhua，ZHANG Dongfang，XIAO Yao，et al. Reinforcement learning-based energy management strategies of fuel cell hybrid vehicles with multi-objective control[J]. Journal of Power Sources，2022，543：231841.
[3] SONG Ruoyang，LIU Xinghua，WEI Zhongbao，et al. Safety and longevity enhanced energy management of fuel cell hybrid electric vehicle with machine learning approach[J]. IEEE Transactions on Transportation Electrification，2024，10(2)：2562-2571.
[4] HAN Ying，LI Qi，WANG Tianhong，et al. Multisource coordination energy management strategy based on SOC consensus for a PEMFC-battery-supercapacitor hybrid tramway[J]. IEEE Transactions on Vehicular Technology，2017，67(1)：296-305.
[5] JIA Chao，QIAO Wei，QU Liyan. A cost-oriented control strategy for energy management of a dual-mode locomotive[C]//2019 IEEE Vehicle Power and Propulsion Conference. Hanoi，Vietnam：IEEE，2019：1-6.
[6] Enang W，Bannister C. Modelling and control of hybrid electric vehicles (A comprehensive review)[J]. Renewable and Sustainable Energy Reviews，2017，74：1210-1239.
[7] YANG Huice，ZHANG Chong，GONG Xun，et al. Energy management strategy of fuel cell hybrid electric vehicle based on dynamic programming[C]//2020 Chinese Automation Congress. Shanghai，China：IEEE，2020：3134-3139.
[8] HU Xiaosong，MURGOVSKI N，JOHANNESSON L M，et al. Optimal dimensioning and power management of a fuel cell/battery hybrid bus via convex programming[J]. IEEE/ASME Transactions on Mechatronics，2014，20(1)：457-468.
[9] 王哲，谢怡，臧鹏飞，等. 基于极小值原理的燃料电池客车能量管理策略[J]. 吉林大学学报(工学版)，2020，50(1)：36-43. WANG Zhe，XIE Yi，ZANG Pengfei，et al. Energy management strategy of fuel cell bus based on Pontryagin’s minimum principle[J]. Journal of Jilin University(Engineering and Technology Edition)，2020，50(1)：36-43.
[10] JIA Chao，QIAO Wei，CUI Junwei，et al. Adaptive model-predictive-control-based real-time energy management of fuel cell hybrid electric vehicles[J]. IEEE Transactions on Power Electronics，2022，38(2)：2681-2694.
[11] 牛超凡. 基于工况预测的燃料电池汽车能量管理策略研究[D]. 长春：吉林大学，2021. NIU Chaofan. Research on energy management strategy of fuel cell vehicle based on condition prediction[D]. Changchun：Jilin University，2021.
[12] ZHANG Yuanzhi，HUANG Zhiyu，ZHANG Caizhi，et al. Improved short-term speed prediction using spatiotemporal-vision-based deep neural network for intelligent fuel cell vehicles[J]. IEEE Transactions on Industrial Informatics，2020，17(9)：6004-6013.
[13] LV Chen，HU Xiaosong，SANGIOVANNI- VINCENTELLI A，et al. Driving-style-based codesign optimization of an automated electric vehicle：A cyber-physical system approach[J]. IEEE Transactions on Industrial Electronics，2018，66(4)：2965-2975.
[14] CUAYAHUITL H，LEE D，RYU S，et al. Ensemble-based deep reinforcement learning for chatbots[J]. Neurocomputing，2019，366：118-130.
[15] BISWAS A，EMADI A. Energy management systems for electrified powertrains：State-of-the-art review and future trends[J]. IEEE Transactions on Vehicular Technology，2019，68(7)：6453-6467.
[16] GUO Liang，LI Zhongliang，OUTBIB R. Reinforcement learning based energy management for fuel cell hybrid electric vehicles[C]//IECON 2021-47th Annual Conference of the IEEE Industrial Electronics Society. Toronto，ON，Canada：IEEE，2021：1-6.
[17] ZHANG Yuxiang，MA Rrui，ZHAO Dongdong，et al. A novel energy management strategy based on dual reward function Q-learning for fuel cell hybrid electric vehicle[J]. IEEE Transactions on Industrial Electronics，2021，69(2)：1537-1547.
[18] WU Jingda，WEI Zhongbao，LI Weihan，et al. Battery thermal-and health-constrained energy management for hybrid electric bus based on soft actor-critic DRL algorithm[J]. IEEE Transactions on Industrial Informatics，2020，17(6)：3751-3761.
[19] WU Jingda，WEI Zhongbao，LIU Kailong，et al. Battery-involved energy management for hybrid electric bus based on expert-assistance deep deterministic policy gradient algorithm[J]. IEEE Transactions on Vehicular Technology，2020，69(11)：12786-12796.
[20] HUANG Ruchen，He Hongwen，Zhao Xuyang，et al. Longevi-ty-aware energy management for fuel cell hybrid electric bus based on a novel proximal policy optimization deep reinforcement learning framework[J]. Journal of Power Sources，2023，561：232717.
[21] PENG Jiankun，ZHANG Siyu，ZHOU Yang，et al. An integrated model for autonomous speed and lane change decision-making based on deep reinforcement learning[J]. IEEE Transactions on Intelligent Transportation Systems，2022，23(11)：21848-21860.
[22] VAJEDI M，AZAD N L. Ecological adaptive cruise controller for plug-in hybrid electric vehicles using nonlinear model predictive control[J]. IEEE Transactions on Intelligent Transportation Systems，2015，17(1)：113-122.
[23] CHADA S K，THOMAS J M，GORGES D，et al. Ecological adaptive cruise control for city buses based on hybrid model predictive control using PnG and traffic light information[C]//2021 IEEE Vehicle Power and Propulsion Conference. Gijon，Spain：IEEE，2021：1-7.
[24] 唐小林，陈佳信，刘腾，等. 基于深度强化学习的混合动力汽车智能跟车控制与能量管理策略研究[J]. 机械工程学报，2021，57(22)：237-246. TANG Xiaolin，CHEN Jiaxin，LIU Teng，et al. Research on deep reinforcement learning-based intelligent car-following control and energy management strategy for hybrid electric vehicles[J]. Journal of Mechanical Engineering，20221，57(2)：237-246.
[25] CHEN Weiqi，YIN Guodong，FAN Yi，et al. Ecological driving strategy for fuel cell hybrid electric vehicle based on continuous deep reinforcement learning[C]//20226th CAA International Conference on Vehicular Control and Intelligence. Nanjing，China：IEEE，2022：1-6.
[26] WEI Zhongbao，QUAN Zhongyi，WU Jingda，et al. Deep deterministic policy gradient-DRL enabled multiphysics- constrained fast charging of lithium-ion battery[J]. IEEE Transactions on Industrial Electronics，2021，69(3)：2588-2598.
[27] 孙向东，李聪，杨欢，等. 基于双向交错并联变换器的电池模拟器研究[J]. 电气工程学报，2024，19(2)：279-287. SUN Xiangdong，LI Cong，YANG Huan，et al. Research on battery simulator based on bidirectional interleaving parallel converter[J]. Journal of Electrical Engineering，2024，19(2)：279-287.
[28] EBESEN S，ELBERT P，GUZZELLA L. Battery state-of-health perceptive energy management for hybrid electric vehicles[J]. IEEE Transactions on Vehicular Technology，2012，61(7)：2893-2900.
[29] HU Xiaosong，ZOU Changfu，TANG Xiaolin，et al. Cost-optimal energy management of hybrid electric vehicles using fuel cell/battery health-aware predictive control[J]. IEEE Transactions on Power Electronics，2020，35(1)：382-392.
[30] KITTERN N，LILL F，KAMMEN M D. Energy storage deployment and innovation for the clean energy transition[J]. Nature Energy，2017，2(9)：649-653.