横纵向耦合跟车场景下基于多智能体深度强化学习的混合动力车队协同能量管理研究

doi:10.3901/JME.2025.02.236

机械工程学报 ›› 2025, Vol. 61 ›› Issue (2): 236-246.doi: 10.3901/JME.2025.02.236

• 运载工程 • 上一篇

扫码分享

横纵向耦合跟车场景下基于多智能体深度强化学习的混合动力车队协同能量管理研究

唐小林¹, 甘炯鹏¹, 张振果²

1. 重庆大学机械与运载工程学院重庆 400044;
2. 上海交通大学机械与动力工程学院上海 200024

收稿日期:2024-01-19 修回日期:2024-09-20 发布日期:2025-02-26
作者简介:唐小林，男，1984年出生，博士，教授，博士研究生导师。主要研究方向为混合动力汽车动力学与节能控制。E-mail：tangxl0923@cqu.edu.cn;甘炯鹏，男，1997年出生。主要研究方向为混合动力汽车能量管理与深度强化学习。E-mail：ganjiongpeng@cqu.edu.cn;张振果(通信作者)，男，1985年出生，博士，副教授，博士研究生导师。主要研究方向为机械系统振动噪声分析与控制。E-mail：zzgjtx@sjtu.edu.cn
基金资助:
国家自然科学基金(52222215，52072051)和重庆市杰出青年基金(2023NSCQ- JQX0009)资助项目。

Research on the Collaborative Energy Management Strategy of Hybrid Electric Vehicle Platoon Based on Multi-agent Deep Reinforcement Learning in the Transverse and Longitudinal Coupled Car-following Scenario

TANG Xiaolin¹, GAN Jiongpeng¹, ZHANG Zhenguo²

1. College of Mechanical and Vehicle Engineering, Chongqing University, Chongqing 400044;
2. School of Mechanical Engineering, Shanghai Jiao Tong University, Shanghai 200024

Received:2024-01-19 Revised:2024-09-20 Published:2025-02-26

摘要/Abstract

摘要： 为了探索多智能体深度强化学习算法在混合动力汽车多目标协同控制中的应用，提出了一种基于多智能体深度确定性策略梯度算法的混合动力车队协同能量管理策略。首先，利用交通仿真软件搭建横纵向耦合跟车场景，以模拟车联网环境实现对车辆信息的准确获取。其次，设计了包含横向变道及纵向跟车的基于规则及网格搜索的横纵向耦合跟车策略，以实现更高的通行效率。最后，利用多智能体深度确定性策略梯度算法设计混合动力车队自适应协同能量管理策略，实现车队整体效益最大化，并通过随机车流初始位置获取随机车队需求功率序列，从而增加策略训练的随机性，提高策略对不同工况的适应性。结果表明，多智能体的车队协同能量管理策略与单智能体相比拥有更好的整体优化效果，并且经随机工况训练后，其工况适应性得到了一定的提升。

关键词: 横纵向耦合跟车, 多智能体深度强化学习, 混合动力车队, 协同能量管理

Abstract: To explore the application of the multi-agent deep reinforcement learning (DRL) algorithm in hybrid electric vehicle multi-objective cooperative control, a multi-agent deep deterministic strategy gradient (MADDPG) algorithm-based hybrid electric vehicle platoon collaborative energy management strategy was proposed. Firstly, the traffic simulation software is used to build a transverse and longitudinal coupled car-following scene to simulate the internet of vehicles environment to achieve accurate acquisition of vehicle information. Secondly, a transverse and longitudinal coupled car-following strategy based on rule and grid search was designed, including lateral lane change and longitudinal car following, to achieve higher traffic efficiency Finally, the MADDPG algorithm was used to design an adaptive collaborative energy management strategy for the hybrid electric vehicle platoon to maximize the overall benefit, and the random vehicle demand power sequence was obtained through the initial position of random traffic flow, thus increasing the randomness of strategy training and improving the adaptability of the strategy to different driving conditions. The results show that the multi-agent vehicle platoon collaborative energy management strategy has a better overall optimization effect than the single agent, and its adaptability to driving conditions has been improved to a certain extent after training in random driving conditions.

Key words: transverse and longitudinal coupled car-following, multi-agent deep reinforcement learning, hybrid electric vehicle platoon, cooperative energy management

中图分类号:

U461

唐小林, 甘炯鹏, 张振果. 横纵向耦合跟车场景下基于多智能体深度强化学习的混合动力车队协同能量管理研究[J]. 机械工程学报, 2025, 61(2): 236-246.

TANG Xiaolin, GAN Jiongpeng, ZHANG Zhenguo. Research on the Collaborative Energy Management Strategy of Hybrid Electric Vehicle Platoon Based on Multi-agent Deep Reinforcement Learning in the Transverse and Longitudinal Coupled Car-following Scenario[J]. Journal of Mechanical Engineering, 2025, 61(2): 236-246.

参考文献

[1] MUSTAFI N N. An overview of hybrid electric vehicle technology[J]. Engines and Fuels for Future Transport，2022：73-102.
[2] 唐小林，陈佳信，高博麟，等. 基于云控系统高精度地图驱动的深度强化学习型混合动力汽车集成控制[J]. 机械工程学报，2022，58(24)：163-177. TANG Xiaolin，CHEN Jiaxin，GAO Baolin，et al. Deep reinforcement learning-based integrated control of hybrid electric vehicles driven by high definition map in cloud control system[J]. Journal of Mechanical Engineering，2022，58(24)：163-177.
[3] DONG P，ZHAO J，LIU X，et al. Practical application of energy management strategy for hybrid electric vehicles based on intelligent and connected technologies：Development stages，challenges，and future trends[J]. Renewable and Sustainable Energy Reviews，2022，170：112947.
[4] GAN Jiongpeng，LI Shen，WEI Chongfeng，et al. Intelligent learning algorithm and intelligent transportation- based energy management strategies for hybrid electric vehicles：A review[J]. IEEE Transactions on Intelligent Transportation Systems，2023，24(10)：10345-10361.
[5] GANESH A H，XU Bin. A review of reinforcement learning based energy management systems for electrified powertrains：progress，challenge，and potential solution[J]. Renewable and Sustainable Energy Reviews，2022，154：111833.
[6] XIE Shaobo，HU Xiaosong，XIN Zongke，et al. Pontryagin’s minimum principle based model predictive control of energy management for a plug-in hybrid electric bus[J]. Applied Energy，2019，236：893-905.
[7] 唐小林，李珊珊，王红，等. 网联环境下基于分层式模型预测控制的车队能量控制策略研究[J]. 机械工程学报，2020，56(14)：119-128. TANG Xiaolin，LI Shanshan，WANG Hong，et al. Research on energy control strategy based on hierarchical model predictive control in connected environment[J]. Journal of Mechanical Engineering，2020，56(14)：119-128.
[8] WU Yuankai，TAN Huachun，PENG Jiankun，et al. Deep reinforcement learning of energy management with continuous control strategy and traffic information for a series-parallel plug-in hybrid electric bus[J]. Applied Energy，2019，247：454-466.
[9] XIONG Shenguang，ZHANG Yishi，WU Chaozhong，et al. Energy management strategy of intelligent plug-in split hybrid electric vehicle based on deep reinforcement learning with optimized path planning algorithm[J]. Proceedings of the Institution of Mechanical Engineers，Part D：Journal of Automobile Engineering，2021，235(14)：3287-3298.
[10] CHEN Jiaxin，SHU Hong，TANG Xiaolin，et al. Deep reinforcement learning-based multi-objective control of hybrid power system combined with road recognition under time-varying environment[J]. Energy，2022，239：122123.
[11] HU Xiaosong，ZHANG Xiaoqian，TANG Xiaolin，et al. Model predictive control of hybrid electric vehicles for fuel economy，emission reductions，and inter-vehicle safety in car-following scenarios[J]. Energy，2020，196：117101.
[12] HOU Shengyan，CHEN Hong，ZHANG Yu，et al. Speed planning and energy management strategy of hybrid electric vehicles in a car-following scenario[J]. Control Theory and Technology，2022，20(2)：185-196.
[13] 唐小林，陈佳信，刘腾，等. 基于深度强化学习的混合动力汽车智能跟车控制与能量管理策略研究[J]. 机械工程学报，2021，57(22)：237-246. TANG Xiaolin，CHEN Jiaxin，LIU Teng，et al. Research on deep reinforcement learning-based intelligent car-following control and energy management strategy for hybrid electric vehicles[J]. Journal of Mechanical Engineering，2021，57(22)：237-246.
[14] OH G，LEBLANC D J，PENG H. Vehicle energy dataset (VED)，a large-scale dataset for vehicle energy consumption research[J]. IEEE Transactions on Intelligent Transportation Systems，2022，23(4)：3302-3312.
[15] LOWE R，WU Y，TAMAR A，et al. Multi-agent actor- critic for mixed cooperative-competitive environments[J]. Proceedings of the 31st International Conference on Neural Information Processing Systems，2017：6382-6393.

横纵向耦合跟车场景下基于多智能体深度强化学习的混合动力车队协同能量管理研究

Research on the Collaborative Energy Management Strategy of Hybrid Electric Vehicle Platoon Based on Multi-agent Deep Reinforcement Learning in the Transverse and Longitudinal Coupled Car-following Scenario

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

本文评价

[1]	杨显通, 郑玲, 金彦林, 张涵柯, 曾迪, 冀杰. 基于车载视觉和动力学模型的路面附着系数融合估计方法[J]. 机械工程学报, 2024, 60(24): 254-264.
[2]	余旭东, 赵剑, 郭烈, 李刚, 黄海波. 考虑接地效应的轮胎柔性圆环建模及其径向振动特性研究[J]. 机械工程学报, 2024, 60(23): 152-163.
[3]	刘赢, 徐利伟, 采国顺, 卢彦博, 殷国栋, 王建强. 通信拓扑结构随机切换及干扰下的多车队列事件驱动控制[J]. 机械工程学报, 2024, 60(22): 276-290.
[4]	唐小林, 杨剑英, 杨凯, 李文博. 面向无信号灯十字路口的自动驾驶多车协同控制方法研究[J]. 机械工程学报, 2024, 60(20): 217-228.
[5]	叶青, 姜笑, 张垚, 汪若尘, 丁仁凯, 蔡英凤. 考虑响应时滞的磁流变半主动悬架H_∞控制与试验研究[J]. 机械工程学报, 2024, 60(18): 276-287.
[6]	钱敏, 耿可可, 殷国栋, 李尚杰, 王子威, 孙宇啸. 基于多元信息融合的车路协同重识别算法[J]. 机械工程学报, 2024, 60(16): 231-240.
[7]	金贤建, 王佳栋, 徐利伟, 严择圆, 卢彦博, 殷国栋, 陈南. 轮毂电机驱动电动汽车主动悬架μ综合鲁棒控制研究[J]. 机械工程学报, 2024, 60(16): 259-269.
[8]	冯吉伟, 殷国栋, 梁晋豪, 庄伟超, 彭湃, 卢彦博, 采国顺, 徐利伟. 考虑介入惩罚因子的智能车辆人机协同控制架构[J]. 机械工程学报, 2024, 60(14): 238-251.
[9]	张伟, 刘辉, 刘金刚, 严鹏飞, 傅兵. 机电复合传动主谐次扰动相位自适应补偿方法[J]. 机械工程学报, 2024, 60(13): 354-365.
[10]	张海伦, 许庆, 高博麟, 王建强, 李克强. 网联环境下驾驶人响应机制——一种交叉路口接近行为定性效应分析与定量驾驶模式提取案例[J]. 机械工程学报, 2024, 60(10): 22-47.
[11]	伍文广, 张斌, 胡林, 张志勇. 人车交互的行人过街方位概率动态预测模型[J]. 机械工程学报, 2024, 60(10): 48-50.
[12]	杨凯, 唐小林, 钟桂川, 王明, 李国法, 胡晓松. 面向无信号灯十字路口场景的自动驾驶安全决策方法研究[J]. 机械工程学报, 2024, 60(10): 147-159.
[13]	褚端峰, 彭赛骞, 胡海洋, 皮大伟. 预见性驾驶风险场模型[J]. 机械工程学报, 2024, 60(10): 160-170.
[14]	胡林, 杨冬兆, 张新, 章杰, 廖家才. 基于DQP-LMPC的智能车超车换道动态路径规划[J]. 机械工程学报, 2024, 60(10): 171-181.
[15]	梁凯冲, 赵治国, 颜丹姝, 赵坤. 基于动态运动基元的车辆高速公路换道轨迹规划[J]. 机械工程学报, 2024, 60(10): 192-206.