Focusing on a continuously variable transmission(CVT) hybrid electric vehicle with integrate starter generator(ISG), modeling and simulating of engine braking was conducted. An optical control strategy of regenerative braking was put forward. When there is no regenerative braking, engine brake force can be used effectively by changing the CVT ratio to reduce the braking force requirements for wheel brakes based on the calculated results of engine braking. When there is regenerative braking, the braking priority sequence is generator braking, engine braking, and finally frictional braking. A forward simulation model for regenerative braking system of hybrid electric vehicle was built. The simulation and analysis under typical driving cycles was conducted. The result shows the control strategy and the models are reasonable and feasible.