In order to solve the problem that requires some factors by manual in the traditional Ncut algorithm, limit the generality of the algorithm, an adaptive image segmentation method is proposed by improving the traditional Ncut algorithm. First, instead of the two control parameters on the calculation of weight matrix that influence the segmentation results in the traditional Ncut algorithm by groups of potential theory; then in order to reduce the sensitive to the number of the cluster and the center of the cluster in the K-means algorithm, calculate on the eigenvector of the Ncut algorithm by the minimum spanning tree, to get the final number of cluster and the center, and then uses the K-means clustering algorithm to get the final segmentation result. The experimental results show that the proposed method not only improves the versatility of the algorithm, and the segmentation is good.