Aiming at the problems of low efficiency, poor detection effect and poor safety of manual detection methods for building exterior wall cracks, a crack detection method based on Unmanned Aerial Vehicle (UAV) aerial photography and computer vision was proposed. Firstly, the UAV was used to collect the crack images by aerial photography around the buildings, and constructed a crack dataset. Secondly, aiming at the problems of discontinuous segmentation of slender cracks, missed detection of cracks and background false detection under complex background in U-Net. The model encoder was replaced with pre-trained ResNet50 to improve the feature expression ability of the model. The improved Atrous Spatial Pyramid Pooling (ASPP) module was added to obtain multi-scale context information. The improved loss function was used to deal with the problem of extremely uneven distribution of positive and negative samples in crack images. Experiments show that the improved U-Net model solved the problems existing in the original model, the IoU and F1_score were increased by 3.53 and 4.18 percentage points respectively. Compared with the classical segmentation model, the improved model has the best crack segmentation performance. Compared with manual detection methods, the proposed method can efficiently, accurately and safely detect building exterior wall cracks.