基于深度强化学习的跟驰场景中燃料电池混合动力汽车多目标能量管理研究

doi:10.3901/JME.2025.12.191

摘要/Abstract

摘要： 为了提高燃料电池混合动力汽车(Fuel cell hybrid electric vehicle,FCHEV)的经济性、动力性与安全性，对跟驰场景下混合动力系统输出功率分配的能量管理问题进行研究。跟驰场景中的FCHEV多目标能量管理研究分为上层自适应巡航控制策略(Adaptive cruise control,ACC)和下层能量管理策略(Energy management strategy,EMS)。所提出的上层ACC策略，通过深度确定性策略梯度(Deep deterministic policy gradient,DDPG)将跟驰安全性、驾驶舒适性和经济性纳入跟驰优化约束范围，控制后车加速度实现对前车的跟驰。所提出的下层EMS策略，将氢气消耗、荷电状态(State of charge,SOC)约束、锂电池(Lithium ion battery,LIB)温度和动力系统寿命控制为优化目标，基于深度强化学习实现多目标的功率优化分配。通过将ACC和EMS分层耦合，其中ACC提供给EMS车辆运行的需求功率，EMS反馈氢气消耗量，可以进一步提高跟驰场景的经济性，降低计算量。仿真结果表明，在所提出的上层跟驰控制下相对车速和跟驰距离误差分别为0.47 m∙s^-1和1.14 m，具有良好的跟随性能；所提出的多目标EMS，相较于单目标策略，氢气消耗降低1.8%，具有更稳定的SOC控制效果，LIB的温升和寿命衰退分别降低了63%和83%，综合经济成本降低了51%；跟驰场景下，ACC与EMS耦合模型与传统ACC相比，氢气消耗降低2.9%，同时保障了跟驰过程的安全性、舒适性和经济性。

关键词: 燃料电池混合动力汽车, 深度强化学习, 跟驰控制, 能量管理

Abstract: In order to improve the economy and safety of fuel cell hybrid electric vehicle(FCHEV), the energy management of hybrid power system output power distribution under car following scenario was studied. The research on the multi-objective energy management of FCHEV under car following scenario was divided into upper adaptive cruise control(ACC) strategy and lower energy management strategy(EMS). The proposed upper ACC strategy takes car following safety, driving comfort and economy into the scope of car following optimization constraints through deep deterministic policy gradient(DDPG), through controlling the acceleration of the rear vehicle to achieve car following of the front vehicle. The proposed lower level EMS strategy takes hydrogen consumption, state of charge(SOC) constraints, temperature of Lithium Ion Battery(LIB) and power system life control as optimization objectives, and realizes multi-objective power optimization allocation based on deep reinforcement learning. By coupling ACC and EMS, in which ACC provides EMS with the required power for vehicle movement, and EMS feeds back the hydrogen consumption, the economy of car following scenario can be further improved and the amount of calculation can be reduced. The simulation results show that under the proposed upper level car following control, the relative speed and following distance errors are 0.47 m∙s^-1 and 1.14 m, respectively, indicating good following performance; The proposed multi-objective EMS can reduce hydrogen consumption by 1.8% compared with the single objective strategy, and has a more stable SOC performance. The LIB temperature rise and life degradation are reduced by 63% and 83% respectively, and the comprehensive economic cost is reduced by 51%; Under car following scenario, compared with the traditional ACC, the proposed ACC-EMS framework further reduces the hydrogen consumption by 2.9% to ensure the safety, comfort and economy of FCHEV during car following process.

Key words: fuel cell hybrid electric vehicle, deep reinforcement learning, car following control, energy management

中图分类号:

U469

曹延旭, 魏中宝, 宋若旸, 何洪文. 基于深度强化学习的跟驰场景中燃料电池混合动力汽车多目标能量管理研究[J]. 机械工程学报, 2025, 61(12): 191-203.

CAO Yanxu, WEI Zhongbao, SONG Ruoyang, HE Hongwen. Research on Multi-objective EMS of Fuel Cell Hybrid Electric Vehicle under Car Following Scenario Based on Deep Reinforcement Learning[J]. Journal of Mechanical Engineering, 2025, 61(12): 191-203.

参考文献

[1] WEI Zhongbao，SONG Ruoyang，JI Dongxu，et al. Hierarchical thermal management for PEM fuel cell with machine learning approach[J]. Applied Thermal Engineering，2024，236：121544.
[2] ZHENG Chunhua，ZHANG Dongfang，XIAO Yao，et al. Reinforcement learning-based energy management strategies of fuel cell hybrid vehicles with multi-objective control[J]. Journal of Power Sources，2022，543：231841.
[3] SONG Ruoyang，LIU Xinghua，WEI Zhongbao，et al. Safety and longevity enhanced energy management of fuel cell hybrid electric vehicle with machine learning approach[J]. IEEE Transactions on Transportation Electrification，2024，10(2)：2562-2571.
[4] HAN Ying，LI Qi，WANG Tianhong，et al. Multisource coordination energy management strategy based on SOC consensus for a PEMFC-battery-supercapacitor hybrid tramway[J]. IEEE Transactions on Vehicular Technology，2017，67(1)：296-305.
[5] JIA Chao，QIAO Wei，QU Liyan. A cost-oriented control strategy for energy management of a dual-mode locomotive[C]//2019 IEEE Vehicle Power and Propulsion Conference. Hanoi，Vietnam：IEEE，2019：1-6.
[6] Enang W，Bannister C. Modelling and control of hybrid electric vehicles (A comprehensive review)[J]. Renewable and Sustainable Energy Reviews，2017，74：1210-1239.
[7] YANG Huice，ZHANG Chong，GONG Xun，et al. Energy management strategy of fuel cell hybrid electric vehicle based on dynamic programming[C]//2020 Chinese Automation Congress. Shanghai，China：IEEE，2020：3134-3139.
[8] HU Xiaosong，MURGOVSKI N，JOHANNESSON L M，et al. Optimal dimensioning and power management of a fuel cell/battery hybrid bus via convex programming[J]. IEEE/ASME Transactions on Mechatronics，2014，20(1)：457-468.
[9] 王哲，谢怡，臧鹏飞，等. 基于极小值原理的燃料电池客车能量管理策略[J]. 吉林大学学报(工学版)，2020，50(1)：36-43. WANG Zhe，XIE Yi，ZANG Pengfei，et al. Energy management strategy of fuel cell bus based on Pontryagin’s minimum principle[J]. Journal of Jilin University(Engineering and Technology Edition)，2020，50(1)：36-43.
[10] JIA Chao，QIAO Wei，CUI Junwei，et al. Adaptive model-predictive-control-based real-time energy management of fuel cell hybrid electric vehicles[J]. IEEE Transactions on Power Electronics，2022，38(2)：2681-2694.
[11] 牛超凡. 基于工况预测的燃料电池汽车能量管理策略研究[D]. 长春：吉林大学，2021. NIU Chaofan. Research on energy management strategy of fuel cell vehicle based on condition prediction[D]. Changchun：Jilin University，2021.
[12] ZHANG Yuanzhi，HUANG Zhiyu，ZHANG Caizhi，et al. Improved short-term speed prediction using spatiotemporal-vision-based deep neural network for intelligent fuel cell vehicles[J]. IEEE Transactions on Industrial Informatics，2020，17(9)：6004-6013.
[13] LV Chen，HU Xiaosong，SANGIOVANNI- VINCENTELLI A，et al. Driving-style-based codesign optimization of an automated electric vehicle：A cyber-physical system approach[J]. IEEE Transactions on Industrial Electronics，2018，66(4)：2965-2975.
[14] CUAYAHUITL H，LEE D，RYU S，et al. Ensemble-based deep reinforcement learning for chatbots[J]. Neurocomputing，2019，366：118-130.
[15] BISWAS A，EMADI A. Energy management systems for electrified powertrains：State-of-the-art review and future trends[J]. IEEE Transactions on Vehicular Technology，2019，68(7)：6453-6467.
[16] GUO Liang，LI Zhongliang，OUTBIB R. Reinforcement learning based energy management for fuel cell hybrid electric vehicles[C]//IECON 2021-47th Annual Conference of the IEEE Industrial Electronics Society. Toronto，ON，Canada：IEEE，2021：1-6.
[17] ZHANG Yuxiang，MA Rrui，ZHAO Dongdong，et al. A novel energy management strategy based on dual reward function Q-learning for fuel cell hybrid electric vehicle[J]. IEEE Transactions on Industrial Electronics，2021，69(2)：1537-1547.
[18] WU Jingda，WEI Zhongbao，LI Weihan，et al. Battery thermal-and health-constrained energy management for hybrid electric bus based on soft actor-critic DRL algorithm[J]. IEEE Transactions on Industrial Informatics，2020，17(6)：3751-3761.
[19] WU Jingda，WEI Zhongbao，LIU Kailong，et al. Battery-involved energy management for hybrid electric bus based on expert-assistance deep deterministic policy gradient algorithm[J]. IEEE Transactions on Vehicular Technology，2020，69(11)：12786-12796.
[20] HUANG Ruchen，He Hongwen，Zhao Xuyang，et al. Longevi-ty-aware energy management for fuel cell hybrid electric bus based on a novel proximal policy optimization deep reinforcement learning framework[J]. Journal of Power Sources，2023，561：232717.
[21] PENG Jiankun，ZHANG Siyu，ZHOU Yang，et al. An integrated model for autonomous speed and lane change decision-making based on deep reinforcement learning[J]. IEEE Transactions on Intelligent Transportation Systems，2022，23(11)：21848-21860.
[22] VAJEDI M，AZAD N L. Ecological adaptive cruise controller for plug-in hybrid electric vehicles using nonlinear model predictive control[J]. IEEE Transactions on Intelligent Transportation Systems，2015，17(1)：113-122.
[23] CHADA S K，THOMAS J M，GORGES D，et al. Ecological adaptive cruise control for city buses based on hybrid model predictive control using PnG and traffic light information[C]//2021 IEEE Vehicle Power and Propulsion Conference. Gijon，Spain：IEEE，2021：1-7.
[24] 唐小林，陈佳信，刘腾，等. 基于深度强化学习的混合动力汽车智能跟车控制与能量管理策略研究[J]. 机械工程学报，2021，57(22)：237-246. TANG Xiaolin，CHEN Jiaxin，LIU Teng，et al. Research on deep reinforcement learning-based intelligent car-following control and energy management strategy for hybrid electric vehicles[J]. Journal of Mechanical Engineering，20221，57(2)：237-246.
[25] CHEN Weiqi，YIN Guodong，FAN Yi，et al. Ecological driving strategy for fuel cell hybrid electric vehicle based on continuous deep reinforcement learning[C]//20226th CAA International Conference on Vehicular Control and Intelligence. Nanjing，China：IEEE，2022：1-6.
[26] WEI Zhongbao，QUAN Zhongyi，WU Jingda，et al. Deep deterministic policy gradient-DRL enabled multiphysics- constrained fast charging of lithium-ion battery[J]. IEEE Transactions on Industrial Electronics，2021，69(3)：2588-2598.
[27] 孙向东，李聪，杨欢，等. 基于双向交错并联变换器的电池模拟器研究[J]. 电气工程学报，2024，19(2)：279-287. SUN Xiangdong，LI Cong，YANG Huan，et al. Research on battery simulator based on bidirectional interleaving parallel converter[J]. Journal of Electrical Engineering，2024，19(2)：279-287.
[28] EBESEN S，ELBERT P，GUZZELLA L. Battery state-of-health perceptive energy management for hybrid electric vehicles[J]. IEEE Transactions on Vehicular Technology，2012，61(7)：2893-2900.
[29] HU Xiaosong，ZOU Changfu，TANG Xiaolin，et al. Cost-optimal energy management of hybrid electric vehicles using fuel cell/battery health-aware predictive control[J]. IEEE Transactions on Power Electronics，2020，35(1)：382-392.
[30] KITTERN N，LILL F，KAMMEN M D. Energy storage deployment and innovation for the clean energy transition[J]. Nature Energy，2017，2(9)：649-653.