There is a lot of non-stationary signals in gear vibration signals，which is analyzed by using Gabor spectrum. To resolve the problem of the choice of window，a method based on entropy is proposed to select the optimal Gabor window width，and the time-frequency resolution of Gabor transform is improved. Since the spectrum has its specific mechanism of crossing-item，the Gabor spectrum is calculated in the region of the key support area and the crossing-item is effectively removed，then the Gabor spectrum is used to analysis the time-frequency distribution of vibration signals of gear，and this method is compared with traditional analysis methods. The result shows that the Gabor spectrum can intuitively react abnormal gear and energy changes with time，and easily diagnose the gear fault.