Abstract:To improve training speed of deep neural networks and reduce resource consumption on edge devices, quantization training methods have been extensively studied. Compared to floating-point or mixed-precision approaches, full integer quantization offers significant potential in edge training scenarios due to its strong hardware compatibility and high computational efficiency. However, conventional integer quantization struggles to adapt to the dynamic changes of data distributions during training, often leading to significant accuracy loss. To address this issue, a data distribution-aware full integer quantization training framework is proposed, which employs a piecewise quantization method to accurately handle long-tailed data distributions and incorporates an adaptive search method to dynamically adjust quantization parameters based on data distributions. Experimental results for training ResNet models on multiple datasets show that the accuracy loss is no more than 2.44% compared to the floating-point training. Compared to the existing integer training methods, the proposed framework reduces the accuracy loss by up to 90.61%. Furthermore, the framework is deployed on an FPGA, and experimental results demonstrate that, compared to the floating-point training framework, the proposed framework saves 27% of memory resources, 53% of DSP computation resources, and reduces execution time by 53%.