基于深度强化学习的锂电池快速充电控制策略

doi:10.3901/JME.2022.22.069

机械工程学报 ›› 2022, Vol. 58 ›› Issue (22): 69-78.doi: 10.3901/JME.2022.22.069

• 特邀专栏:车载电化学能源系统 • 上一篇下一篇

扫码分享

基于深度强化学习的锂电池快速充电控制策略

唐鑫¹, 欧阳权¹, 黄俍卉², 王志胜¹, 马瑞¹

1. 南京航空航天大学自动化学院南京 211106;
2. 浙江科技学院自动化与电气工程学院杭州 310023

收稿日期:2022-02-07 修回日期:2022-10-08 出版日期:2022-11-20 发布日期:2023-02-07
通讯作者: 欧阳权(通信作者),男,1991年出生,博士,副教授,硕士研究生导师。主要研究方向为锂电池管理系统、新能源系统集成与控制。E-mail:ouyangquan@nuaa.edu.cn
作者简介:唐鑫,男,1998年出生。主要研究方向为控制系统工程与应用。E-mail:tangxin@nuaa.edu.cn;黄俍卉,女,1991年出生,博士,讲师。主要研究方向为新能源系统集成与控制、燃料电池建模与控制。E-mail:120036@zust.edu.cn;王志胜,男,1970年出生,博士,教授,博士研究生导师。主要研究方向为智能机器人技术,锂电池管理系统。E-mail:wangzhisheng@nuaa.edu.cn;马瑞,男,1996年出生,硕士研究生。主要研究方向为锂电池充电控制。E-mail:maruinuaa@nuaa.edu.cn
基金资助:
国家自然科学基金(61903189);中国博士后科学基金(2020M681589);中央高校基本科研业务费(NS2021023)资助项目

Fast Charging Control for Lithium-ion Batteries Based on Deep Reinforcement Learning

TANG Xin¹, OUYANG Quan¹, HUANG Lang-hui², WANG Zhi-sheng¹, MA Rui¹

1. College of Automation Engineering, Nanjing University of Aeronautics and Astronautics, Nanjing 211106;
2. School of Automation and Electrical Engineering, Zhejiang University of Science and Technology, Hangzhou 310023

Received:2022-02-07 Revised:2022-10-08 Online:2022-11-20 Published:2023-02-07

摘要/Abstract

摘要： 安全高效的锂电池充电控制策略对于电动汽车的发展具有重要推动作用。针对锂电池的快速充电问题,提出一种综合考虑锂电池充电速度、能量损耗、安全约束多目标优化充电控制策略。基于动作-评价网络框架,利用基于近端策略优化的深度强化学习算法,训练出使得充电目标对应的奖励函数最大的充电策略神经网络和策略评估神经网络。然后,利用训练完成的充电策略神经网络根据当前电价和电池SOC智能决策出最优的充电电流。该充电控制策略的优势在于能够在保证快速充电的同时,实现充电花费最小化。同时,充电策略神经网络在线运算量较小,与基于模型的在线优化算法相比更能满足充电控制的实时性要求。最后,仿真结果表明,该充电控制策略与传统恒流-恒压法相比,具有兼顾充电速度与电费支出的优势,满足快速充电任务需求的同时,最高可降低25%的充电成本。

关键词: 快速充电控制, 充电成本优化, 深度强化学习, 锂电池

Abstract: Safe and efficient lithium battery charging control strategy plays an important role in promoting the development of electric vehicles. Aiming at the problem of fast charging of lithium batteries, a multi-objective optimal charging control strategy is proposed that comprehensively considers the charging speed, energy consumption and safety constraints of lithium batteries. Based on the action-evaluation network framework, a deep reinforcement learning algorithm based on proximal policy optimization is used to train a charging policy neural network and a policy evaluation neural network that maximize the reward function corresponding to the charging target. Then, using the trained charging strategy neural network to intelligently decide the optimal charging current according to the current electricity price and battery SOC. The advantage of this charging control strategy is that it can minimize charging costs while ensuring fast charging. At the same time, the online calculation amount of the charging strategy neural network is small, and compared with the model-based online optimization algorithm, it can better meet the real-time requirements of charging control. Finally, the simulation results show that, compared with the traditional constant current-constant voltage method, the charging control strategy has the advantages of taking into account the charging speed and electricity expenditure, and can reduce the charging cost by up to 25% while meeting the requirements of fast charging tasks.

Key words: fast charging control algorithm, charging cost optimization, deep reinforcement learning, lithium-ion batteries

中图分类号:

TG156

唐鑫, 欧阳权, 黄俍卉, 王志胜, 马瑞. 基于深度强化学习的锂电池快速充电控制策略[J]. 机械工程学报, 2022, 58(22): 69-78.

TANG Xin, OUYANG Quan, HUANG Lang-hui, WANG Zhi-sheng, MA Rui. Fast Charging Control for Lithium-ion Batteries Based on Deep Reinforcement Learning[J]. Journal of Mechanical Engineering, 2022, 58(22): 69-78.

参考文献

[1] ANDWARI A M,PESIRIDIS A,RAJOO S,et al. A review of battery electric vehicle technology and readiness levels[J]. Renewable and Sustainable Energy Reviews,2017,78:414-430.
[2] 陈莹.电动车用锂离子电池快速充电技术研究[D].无锡:江南大学,2018.CHEN Ying. Research on fast charging technology of lithium-ion batteries for electric vehicles[D]. Wuxi:Jiangnan University,2018.
[3] LIU K,LI K,PENG Q,et al. A brief review on key technologies in the battery management system of electric vehicles[J]. Frontiers of Mechanical Engineering,2019,14(1):47-64.
[4] ANDREA D. Battery management systems for large lithium-ion battery packs[M]. Fitchburg:Artech House,2010.
[5] ZHANG S S. The effect of the charging protocol on the cycle life of a Li-ion battery[J]. Journal of Power Sources,2006,161(2):1385-1391.
[6] PATNAIK L,PRANEETH A,WILLIAMSON S S. A closed-loop constant-temperature constant-voltage charging technique to reduce charge time of lithium-ion batteries[J]. IEEE Transactions on Industrial Electronics,2018,66(2):1059-1067.
[7] WANG S C,LIU Y H. A PSO-based fuzzy-controlled searching for the optimal charge pattern of Li-ion batteries[J]. IEEE Transactions on Industrial Electronics,2014,62(5):2983-2993.
[8] ABDOLLAHI A,HAN X,AVVARI G V,et al. Optimal battery charging,Part I:Minimizing time-to-charge,energy loss,and temperature rise for OCV-resistance battery model[J]. Journal of Power Sources,2016,303:388-398.
[9] OUYANG Q,WANG Z,LIU K,et al. Optimal charging control for lithium-ion battery packs:A distributed average tracking approach[J]. IEEE Transactions on Industrial Informatics,2019,16(5):3430-3438.
[10] LIU K,ZOU C,LI K,et al. Charging pattern optimization for lithium-ion batteries with an electrothermal-aging model[J]. IEEE Transactions on Industrial Informatics,2018,14(12):5463-5474.
[11] OUYANG Q,XU G,FANG H,et al. Fast charging control for battery packs with combined optimization of charger and equalizers[J]. IEEE Transactions on Industrial Electronics,2021,68(11):11076-11086.
[12] CHANG F,CHEN T,SU W,et al. Charging control of an electric vehicle battery based on reinforcement learning[C]//International Renewable Energy Congress(IREC). IEEE,2019:1-63.
[13] WEI Z,QUAN Z,WU J,et al. Deep deterministic policy gradient-DRL enabled multiphysics-constrained fast charging of lithium-ion battery[J]. IEEE Transactions on Industrial Electronics,2022,69(3):2588-2598.
[14] ZOU C, HU X, WEI Z, et al. Electrothermal dynamics-conscious lithium-ion battery cell-level charging management via state-monitored predictive control[J]. Energy,2017,141:250-259.
[15] LIU K,ZOU C,LI K,et al. Charging pattern optimization for lithium-ion batteries with an electrothermal-aging model[J]. IEEE Transactions on Industrial Informatics,2018,14(12):5463-5474.
[16] KIM Y,MOHAN S,SIEGEL J B,et al. The estimation of temperature distribution in cylindrical battery cells under unknown cooling conditions[J]. IEEE Transactions on Control Systems Technology,2014,22(6):2277-2286.
[17] CHUNG C H,JANGRA S,LAI Q,et al. Optimization of electric vehicle charging for battery maintenance and degradation management[J]. IEEE Transactions on Transportation Electrification,2020,6(3):958-969.
[18] SU S,LI H,GAO D W. Optimal planning of charging for plug-in electric vehicles focusing on users' benefits[J].Energies,2017,10(7):952.
[19] KONDA V,TSITSIKLIS J. Actor-critic algorithms[C]//Advances in Neural Information Processing Systems,2000:1008-1014.
[20] LIU B, CAI Q, YANG Z, et al. Neural trust region/proximal policy optimization attains globally optimal policy[C]//Advances in Neural Information Processing Systems,2019:32.
[21] LIU K,HU X,YANG Z,et al. Lithium-ion battery charging management considering economic costs of electrical energy loss and battery degradation[J]. Energy Conversion and Management,2019,195:167-179.
[22] CAO Y,TANG S,LI C,et al. An optimized EV charging model considering TOU price and SOC curve[J]. IEEE Transactions on Smart Grid,2011,3(1):388-393.

基于深度强化学习的锂电池快速充电控制策略

Fast Charging Control for Lithium-ion Batteries Based on Deep Reinforcement Learning

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

本文评价

[1]	王无印, 黄子钊, 庄子龙, 方怀瑾, 秦威. 基于深度强化学习的自动化码头堆场场桥调度方法[J]. 机械工程学报, 2024, 60(6): 44-57.
[2]	郭景华, 李文昌, 王班, 王靖瑶. 基于深度强化学习的网联混合动力汽车队列控制[J]. 机械工程学报, 2024, 60(2): 262-271.
[3]	于全庆, 王灿, 李建明, 汤爱华, 赵立军. 多拓扑结构锂电池组外短路特性分析及模型评价[J]. 机械工程学报, 2023, 59(6): 159-172.
[4]	高锋阳, 张浩然, 王文祥, 李明明. 规则控制与行驶工况相结合的现代有轨电车能量管理策略[J]. 机械工程学报, 2023, 59(4): 221-231.
[5]	钟沛成, 骆德渊, 庞明君. 基于深度强化学习的四足机器人跟随策略研究及系统实现[J]. 机械工程学报, 2023, 59(13): 79-88.
[6]	陈睿奇, 黎雯馨, 王传洋, 杨宏兵. 基于深度强化学习的工序交互式智能体Job shop调度方法[J]. 机械工程学报, 2023, 59(12): 78-88.
[7]	李维聪, 穆浩, 沈恒龙, 于全庆. 固态锂电池在载运工具中的应用前景分析^*[J]. 电气工程学报, 2022, 17(4): 88-102.
[8]	于子轩, 孟国栋, 谢小军, 赵勇, 成永红. 磷酸铁锂储能电池过充热失控仿真研究^*[J]. 电气工程学报, 2022, 17(3): 30-39.
[9]	唐小林, 陈佳信, 高博麟, 杨凯, 胡晓松, 李克强. 基于云控系统高精度地图驱动的深度强化学习型混合动力汽车集成控制[J]. 机械工程学报, 2022, 58(24): 163-177.
[10]	洪琰, 周龙, 张俊, 李相俊, 汪湘晋, 马瑜涵, 郑岳久. 锂离子电池充放电机械压力特性研究[J]. 机械工程学报, 2022, 58(20): 410-420.
[11]	王辉, 徐佳文, 严如强. 基于多尺度注意力深度强化学习网络的行星齿轮箱智能诊断方法[J]. 机械工程学报, 2022, 58(11): 133-142.
[12]	张朝龙, 赵筛筛, 何怡刚. 基于信息熵与PSO-LSTM的锂电池组健康状态估计方法[J]. 机械工程学报, 2022, 58(10): 180-190.
[13]	唐小林, 陈佳信, 刘腾, 李佳承, 胡晓松. 基于深度强化学习的混合动力汽车智能跟车控制与能量管理策略研究[J]. 机械工程学报, 2021, 57(22): 237-246.
[14]	李志杰, 陈吉清, 兰凤崇, 周云郊. 方形锂离子电池内芯层集式模型方法及细观变形失效分析[J]. 机械工程学报, 2021, 57(18): 229-239.
[15]	王怀铷, 孙宜听, 金阳. 磷酸铁锂储能电池簇过充热失控蔓延特性仿真研究[J]. 机械工程学报, 2021, 57(14): 32-39.