For multiclass obstacles recognition for intelligent vehicle in urban traffic scenes, an improved Binary Tree Support Vector Machine (BTSVM) based on ensemble learning is presented. Based on the distributing probability and pattern diversity of each obstacle in urban traffic scenes, a compatible tree structure of BTSVM is designed. An approach based on AdaBoost ensemble learning is applied to reduce the transfer error and improve the accuracy and generalization ability of pernode classifier. The proposed method can efficiently recognize 6 kinds of normal obstacle patterns in urban traffic scenes.