To improve the fuel economy and control strategy stability of hybrid electric vehicles (HEVs), with taking the third-generation Prius hybrid electric vehicle as the research object, a hierarchical energy management strategy is created by combining an equivalent fuel consumption minimization strategy (ECMS) with a deep reinforcement learning (DRL) method. The simulation results show that the hierarchical control strategy not only enables the agent in reinforcement learning to achieve adaptive energy-saving control without a model, but also ensures that the state of charge (SOC) of the hybrid vehicle meets the constraints under all operating conditions. Compared with the rule-based energy management strategy, this layered control strategy improves the fuel economy by 20.83% to 32.66%. Additionally, increasing the prediction information of the vehicle speed by the agent further reduces the fuel consumption by about 5.12%. Compared with the deep reinforcement learning strategy alone, this combined strategy improves fuel economy by about 8.04%. Furthermore, compared with the A-ECMS strategy that uses SOC offset penalty, the fuel economy is improved by 5.81% to 16.18% under this proposed strategy.