Severe torque on bit fluctuations appears when the bit penetrates heterogeneous formations especially at depth and ultra-deep drilling. As a results, the excited torsional vibration in the bottom-hole assembly often leads to stick-slip and bit-stopping, which challenges the safe and integrity of the drill string apparatus. In this paper, a bit-torque damping tool that self-adapts to rock-breaking torque is developed to solve the problem, and a mathematical model of the tool is established. The tool dynamics model is coupled to the drill-string dynamics model, and bit-torque responses are analyzed for conventional, compound, and percussion drilling. Simulation analyses are performed on key parameters that govern tool performance. The results show that the adaptive torque tool greatly reduces bit-torque fluctuations under above three conditions, and the reduction performs best for high-frequency torque oscillations. Further optimization of the rotor inertia and torsional-spring stiffness remarkably improves the torsional vibration damping effect.