Based on the continuity and relativity in time and space of video surveillance sequences, similar face areas were found in motion areas using symmetrical frame differences of video sequences and the clustering of skincolor features. By improving the algorithm for face locating and using geometric facial features, multiple faces were detected in complex video sequences. The faces in video sequences were predicted via motion quotiety and horizontal and vertical adjustment factors. The experimental results show that this algorithm has lower complexity, higher veracity, and better robustness in face detection in situations with changing gestures, expressions and backgrounds. The prediction and tracking effects are satisfied.