Considering the three-dimensional inhomogeneity of soil surrounding the pile, the response of non-uniform viscoelastic pile under axial dynamic loading is studied. Firstly, combining the boundary condition, the complex stiffness of axial different soil layers are obtained by the complex stiffness transfer model of radial multi-zone plane strain. Then, the analytical solution of dynamic response at pile top in frequency domain is proposed by solving the dynamic equation of non-uniform viscoelastic pile section one by one from the bottom up to the top, and the relevant semi-analytical solution response in time domain is adopted by the convolution theorem and the Inverse Fourier Transform. At last, the effect of parameters of pile and soil are investigated to get the nature of the dynamic response of pile top in frequency domain and time domain.