The soil around the pipe pile and pile core soil are regarded as viscoelastic medium, and the stress-strain relationship for them are described by fractional derivative Kelvin viscoelastic constitutive model. The torsional vibrations are solved by Fourier transformation and separation variable method by considering the circumferential displacement of the soil only. Considering the forces acting on the pipe piles, the torsional vibration in the fractional derivative Kelvin viscoelastic soil is established. The torsional complex stiffness at pipe pile head is obtained by solving the torsional vibration of the pipe pile. The results show that the model soil parameters α₁ and T_b1 have certain influence on the torsional vibration while the influence of the pile core soil model parameters α₂ and T_b2 is related to frequency. The curves of real and imaginary parts of torsional complex stiffness with frequency fluctuate more greatly when the shear modulus ratio μ is larger and μ<1, and the influence of shear modulus ratio μ on the torsional vibration of pipe pile is very small when μ>1. Wall thickness, length diameter ratio of pipe pile, as well as the shear modulus of pipe pile and soil have great influence on the torsional vibration of pipe pile.