In order to solve the problem of target loss due to size change in visual tracking, an improved particle filter vehicle tracking algorithm fusing visual and millimeter wave radar is proposed. First, genetic algorithm is used to improve particle degradation and resampling induced by standard particle filtering. The dynamic adaptive genetic cross probability is calculated according to particle degradation degree, and instead of mean distribution Gaussian function is used to calculate fitness. Then, the HSV histogram features are combined with the improved particle to achieve vehicle multi-target tracking. Finally, the location and size of the tracking bounding boxes are modified by the range information from radar. The experimental results show that compared with the standard particle filter, the improved particle filter algorithm significantly improves the multi-object tracking accuracy (MOTA) and multi-object tracking precision (MOTP) by 22.1% and 21.1% respectively. Compared to visual tracking algorithms, the tracking algorithm that fused radar data can improve the precision by 9.2% again.