Fine-grained image classification is a more detailed division of the sub-categories of traditional image classification, which achieves a more sophisticated identification of objects. And it is a very challenging research in the field of computer vision. By analyzing the existing fine-grained image classification algorithm and Xception model, we propose to apply the Xception model to the fine-grained image classification task. Initialization of convolution layers uses pre-training model parameters of ImageNet classification. Then we resize images, transform data type, normalize value, and randomly initialize classifier. Finally, the network is fine-tuned. Our method obtains 71.0%, 89.9% and 91.4% per-image accuracy on the CUB200-2011, Flower102 and Stanford Dogs dataset respectively. The experimental results show that the Xception model has good generalization ability in fine-grained image classification. Because it does not need additional annotation information such as object bounding box and part annotation, the Xception model has good versatility and robustness in fine-grained image classification.