Machine learning algorithms are playing a more important role in building energy consumption prediction during the conceptual design. The selection of the machine learning algorithms and parameter setting have become a focus in the field of building performance design. However, the algorithms and their parameters are usually determined by the principle of algorithms rather than the features of the training samples which also have an effect on the performance of algorithms. Therefore, a classification method based on the quality of training samples which is evaluated by sample size and sample distribution characteristics is proposed. The performance of different machine learning algorithms for different quality sample sets is tested, and algorithm selection and parameter setting strategies for different quality sample sets are formulated. The relationship between sample quality and algorithm performance is investigated to provide effective guidance for architects.