Research on the Collaborative Energy Management Strategy of Hybrid Electric Vehicle Platoon Based on Multi-agent Deep Reinforcement Learning in the Transverse and Longitudinal Coupled Car-following Scenario

doi:10.3901/JME.2025.02.236

Abstract

Abstract: To explore the application of the multi-agent deep reinforcement learning (DRL) algorithm in hybrid electric vehicle multi-objective cooperative control, a multi-agent deep deterministic strategy gradient (MADDPG) algorithm-based hybrid electric vehicle platoon collaborative energy management strategy was proposed. Firstly, the traffic simulation software is used to build a transverse and longitudinal coupled car-following scene to simulate the internet of vehicles environment to achieve accurate acquisition of vehicle information. Secondly, a transverse and longitudinal coupled car-following strategy based on rule and grid search was designed, including lateral lane change and longitudinal car following, to achieve higher traffic efficiency Finally, the MADDPG algorithm was used to design an adaptive collaborative energy management strategy for the hybrid electric vehicle platoon to maximize the overall benefit, and the random vehicle demand power sequence was obtained through the initial position of random traffic flow, thus increasing the randomness of strategy training and improving the adaptability of the strategy to different driving conditions. The results show that the multi-agent vehicle platoon collaborative energy management strategy has a better overall optimization effect than the single agent, and its adaptability to driving conditions has been improved to a certain extent after training in random driving conditions.

Key words: transverse and longitudinal coupled car-following, multi-agent deep reinforcement learning, hybrid electric vehicle platoon, cooperative energy management

CLC Number:

U461

TANG Xiaolin, GAN Jiongpeng, ZHANG Zhenguo. Research on the Collaborative Energy Management Strategy of Hybrid Electric Vehicle Platoon Based on Multi-agent Deep Reinforcement Learning in the Transverse and Longitudinal Coupled Car-following Scenario[J]. Journal of Mechanical Engineering, 2025, 61(2): 236-246.

References

[1] MUSTAFI N N. An overview of hybrid electric vehicle technology[J]. Engines and Fuels for Future Transport，2022：73-102.
[2] 唐小林，陈佳信，高博麟，等. 基于云控系统高精度地图驱动的深度强化学习型混合动力汽车集成控制[J]. 机械工程学报，2022，58(24)：163-177. TANG Xiaolin，CHEN Jiaxin，GAO Baolin，et al. Deep reinforcement learning-based integrated control of hybrid electric vehicles driven by high definition map in cloud control system[J]. Journal of Mechanical Engineering，2022，58(24)：163-177.
[3] DONG P，ZHAO J，LIU X，et al. Practical application of energy management strategy for hybrid electric vehicles based on intelligent and connected technologies：Development stages，challenges，and future trends[J]. Renewable and Sustainable Energy Reviews，2022，170：112947.
[4] GAN Jiongpeng，LI Shen，WEI Chongfeng，et al. Intelligent learning algorithm and intelligent transportation- based energy management strategies for hybrid electric vehicles：A review[J]. IEEE Transactions on Intelligent Transportation Systems，2023，24(10)：10345-10361.
[5] GANESH A H，XU Bin. A review of reinforcement learning based energy management systems for electrified powertrains：progress，challenge，and potential solution[J]. Renewable and Sustainable Energy Reviews，2022，154：111833.
[6] XIE Shaobo，HU Xiaosong，XIN Zongke，et al. Pontryagin’s minimum principle based model predictive control of energy management for a plug-in hybrid electric bus[J]. Applied Energy，2019，236：893-905.
[7] 唐小林，李珊珊，王红，等. 网联环境下基于分层式模型预测控制的车队能量控制策略研究[J]. 机械工程学报，2020，56(14)：119-128. TANG Xiaolin，LI Shanshan，WANG Hong，et al. Research on energy control strategy based on hierarchical model predictive control in connected environment[J]. Journal of Mechanical Engineering，2020，56(14)：119-128.
[8] WU Yuankai，TAN Huachun，PENG Jiankun，et al. Deep reinforcement learning of energy management with continuous control strategy and traffic information for a series-parallel plug-in hybrid electric bus[J]. Applied Energy，2019，247：454-466.
[9] XIONG Shenguang，ZHANG Yishi，WU Chaozhong，et al. Energy management strategy of intelligent plug-in split hybrid electric vehicle based on deep reinforcement learning with optimized path planning algorithm[J]. Proceedings of the Institution of Mechanical Engineers，Part D：Journal of Automobile Engineering，2021，235(14)：3287-3298.
[10] CHEN Jiaxin，SHU Hong，TANG Xiaolin，et al. Deep reinforcement learning-based multi-objective control of hybrid power system combined with road recognition under time-varying environment[J]. Energy，2022，239：122123.
[11] HU Xiaosong，ZHANG Xiaoqian，TANG Xiaolin，et al. Model predictive control of hybrid electric vehicles for fuel economy，emission reductions，and inter-vehicle safety in car-following scenarios[J]. Energy，2020，196：117101.
[12] HOU Shengyan，CHEN Hong，ZHANG Yu，et al. Speed planning and energy management strategy of hybrid electric vehicles in a car-following scenario[J]. Control Theory and Technology，2022，20(2)：185-196.
[13] 唐小林，陈佳信，刘腾，等. 基于深度强化学习的混合动力汽车智能跟车控制与能量管理策略研究[J]. 机械工程学报，2021，57(22)：237-246. TANG Xiaolin，CHEN Jiaxin，LIU Teng，et al. Research on deep reinforcement learning-based intelligent car-following control and energy management strategy for hybrid electric vehicles[J]. Journal of Mechanical Engineering，2021，57(22)：237-246.
[14] OH G，LEBLANC D J，PENG H. Vehicle energy dataset (VED)，a large-scale dataset for vehicle energy consumption research[J]. IEEE Transactions on Intelligent Transportation Systems，2022，23(4)：3302-3312.
[15] LOWE R，WU Y，TAMAR A，et al. Multi-agent actor- critic for mixed cooperative-competitive environments[J]. Proceedings of the 31st International Conference on Neural Information Processing Systems，2017：6382-6393.