混合动力汽车深度强化学习分层能量管理策略
作者:
作者单位:

1.重庆大学;2.重庆大学 机械与运载工程学院

中图分类号:

U471.15???????

基金项目:

新能源汽车能量管理测试评价与能耗集成开发应用、车辆动力传动系统功率瞬变过程动力学与系统分层设计方法研究


Deep Reinforcement Learning Hierarchical Energy Management Strategy for Hybrid Electric Vehicles
Author:
Affiliation:

1.Chongqing University;2.Chongqing University,College of Mechanical and Vehicle Engineering

Fund Project:

New energy vehicle energy management test evaluation and energy consumption integrated development and application、Research on Dynamics and System Hierarchical Design Method of Vehicle Powertrain Power Transient Process

  • 摘要
  • | |
  • 访问统计
  • |
  • 参考文献 [18]
  • | | | |
  • 文章评论
    摘要:

    为了提高混合动力汽车的燃油经济性和控制策略的稳定性,以第三代普锐斯混联式混合动力汽车作为研究对象,提出了一种等效燃油消耗最小策略(ECMS)与深度强化学习方法(DRL)结合的分层能量管理策略。仿真结果证明,该分层控制策略不仅可以让强化学习中的智能体在无模型的情况下实现自适应节能控制,同时也能保证该混合动力汽车在所有工况下的SOC都满足约束限制。与基于规则的能量管理策略相比,此分层控制策略可以将燃油经济性提高20.83%-32.66%;增加智能体对车速的预测信息,可进一步降低约5.12%的燃油消耗;与没有分层的深度强化学习策略相比,此策略可将燃油经济性提高约8.04%;与使用SOC偏移惩罚的自适应等效燃油消耗最小策略(A-ECMS)相比,此策略下的燃油经济性将提高5.81%-16.18%。

    Abstract:

    In order to improve the fuel economy and control strategy stability of HEVs, taking the third-generation Prius hybrid electric vehicle as the research object, an equivalent fuel consumption minimization strategy (ECMS) and a deep reinforcement learning method ( DRL) combined hierarchical energy management strategy. The simulation results show that the hierarchical control strategy can not only enable the agent in reinforcement learning to achieve adaptive energy-saving control without model, but also ensure that the SOC of the hybrid vehicle meets the constraints under all operating conditions. Compared with the rule-based energy management strategy, this layered control strategy can improve the fuel economy by 20.83%-32.66%; increasing the prediction information of the vehicle speed by the agent can further reduce the fuel consumption by about 5.12%; Compared with the deep reinforcement learning strategy of this strategy can improve the fuel economy by about 8.04%; compared with the A-ECMS strategy using SOC offset penalty, the fuel economy under this strategy will be improved by 5.81%-16.18%.

    参考文献
    [1] Onori S, Serrao L, Rizzoni G. Hybrid Electric Vehicles Energy Management Strategies[M]. London: Springer London, 2016.
    [2] Scordia J, Renaudin M D, Trigui R, et al. Global optimisation of energy management laws in hybrid vehicles using dynamic programming[J]. International Journal of Vehicle Design, Inderscience Publishers, 2005, 39(4): 349.
    [3] Liu J, Peng H. Control optimization for a power-split hybrid vehicle[A]. 2006 American Control Conference[C]. 2006: 6 pp.-.
    [4] Musardo C, Rizzoni G, Guezennec Y, et al. A-ECMS: An adaptive algorithm for hybrid electric vehicle energy management[J]. European Journal of Control, Elsevier, 2005, 11(4–5): 509–524.
    [5] Serrao L, Onori S, Rizzoni G. ECMS as a realization of Pontryagin’s minimum principle for HEV control[A]. 2009 American Control Conference[C]. 2009: 3964–3969.
    [6] Rezaei A. An Optimal Energy Management Strategy for Hybrid Electric Vehicles[D]. Houghton, Michigan: Michigan Technological University, 2017.
    [7] Sun C, Sun F, He H. Investigating adaptive-ECMS with velocity forecast ability for hybrid electric vehicles[J]. Applied Energy, Elsevier BV, 2017, 185: 1644–1653.
    [8] Hu X, Liu T, Qi X, et al. Reinforcement learning for Hybrid and Plug-In Hybrid Electric Vehicle Energy Management Recent Advances and Prospects[J]. Ieee Industrial Electronics Magazine, Piscataway: Ieee-Inst Electrical Electronics Engineers Inc, 2019, 13(3): 16–25.
    [9] Zhao P, Wang Y, Chang N, et al. A deep reinforcement learning framework for optimizing fuel economy of hybrid electric vehicles[A]. 2018, 2018-January: 196–202.
    [10] Lian R, Peng J, Wu Y, et al. Rule-interposing deep reinforcement learning based energy management strategy for power-split hybrid electric vehicle[J]. Energy, 2020, 197: 117297.
    [11] Hu Y, Li W, Xu K, et al. Energy Management Strategy for a Hybrid Electric Vehicle Based on Deep Reinforcement Learning[J]. Applied Sciences, Multidisciplinary Digital Publishing Institute, 2018, 8(2): 187.
    [12] Wang Y, Tan H, Wu Y, et al. Hybrid Electric Vehicle Energy Management With Computer Vision and Deep Reinforcement Learning[J]. IEEE Transactions on Industrial Informatics, 2021, 17(6): 3857–3868.
    [13] Qi X, Luo Y, Wu G, et al. Deep reinforcement learning-based vehicle energy efficiency autonomous learning system[A]. 2017 IEEE Intelligent Vehicles Symposium (IV)[C]. 2017: 1228–1233.
    [14] Zhang Z, Zhang D, Qiu R C. Deep reinforcement learning for power system applications: An overview[J]. CSEE Journal of Power and Energy Systems, CSEE, 2019, 6(1): 213–225.
    [15] 余志生. 汽车理论[M]. 第五版. 北京: 机械工业出版社, 2009.Yu Z S. Vehicle Dynamic[M]. Fifth edition. Beijing: China Machine Press, 2009.(in Chinese)
    [16] Paganelli G. Conception et commande d’une chaîne de traction pour véhicule hybride parallèle thermique et électrique[D]. Valenciennes, 1999.
    [17] Paganelli G. A general formulation for the instantaneous control of the power split in charge-sustaining hybrid electric vehicles[A]. Proc. AVEC 2000, 5th Int. Symp. on Advanced Vehicle Control[C]. 2000.
    [18] Onori S, Serrao L, Rizzoni G. Adaptive equivalent consumption minimization strategy for hybrid electric vehicles[A]. Dynamic Systems and Control Conference[C]. 2010, 44175: 499–505.
    相似文献
    引证文献
    网友评论
    网友评论
    分享到微博
    发 布
引用本文
分享
文章指标
  • 点击次数:
  • 下载次数:
  • HTML阅读次数:
  • 引用次数:
历史
  • 收稿日期:2022-02-28
  • 最后修改日期:2022-05-12
  • 录用日期:2022-05-13
文章二维码