Currently, the traditional grid density algorithm for building facade extraction failures to consider interferences from ground targets around the buildings, and has a low adaptivity to buildings in different scences. Aiming to resolve these weakness, this paper considered various typical ground objects in the building area and their local and global spatial characteristics, which are described by single point semantics, grid semantics and regional semantics, forming a multi-level semantic feature descriptor. Based on this descriptor, a multi-level semantic feature extraction method was proposed to extract the building facade point cloud. The experimental results show that this algorithm can be used to quickly and accurately extract the building facades of low, high and super high buildings from point clouds. Overall, this algorithm achieves a high precision, a high efficiency and a good adaptability.