Aiming at the shortcoming of low recognition and training speed in embedded speaker - recognition system based on DS Phard-core processor, a new scheme of system based on FPGA and vector quantization principle is presented. In the speaker-recognition system based on the vector quantization and generation algorithm, a fitness parallel process hardware structure is presented by the scheme which consumes much less time than software processing while getting the fitness. The test shows that the system uses this method obtains both high recognition rate and higher speed in training and recognition.