In order to reduce the plastic seismic damage of prefabricated pier with grouting sleeve and resume the function of the pier in a timely manner, the concept of the prefabricated pier with grouting sleeve and replaceable damper was proposed. A conventional prefabricated pier with grouting sleeve and a pier with grouting sleeve and replaceable damper were analyzed using numerical modeling method. Two models were established to compare the seismic performance between the conventional pier and the bridge pier with replaceable damper. The result shows that the horizontal bearing capacity and cumulative energy dissipation of the prefabricated pier with replaceable damper are 10% and 27% higher than conventional bridge pier, respectively. The maximum residual displacement of the prefabricated pier with replaceable damper decreases by 19%. The damper can effectively control the structural dynamic responses, reducing the prefabricated pier damage.