The performance of asphalt mixture is closely related with microscopic structure. Discrete element method (DEM) has the great advantage in solving microscopic problem of the asphalt mixture with complex, multiphase and discontinuous medium materials. To solve the problem the past DEM could not perfectly simulate aggregate particle shape in HMA, a geometric modeling method of DEM based on the digital image processing was established. Firstly, the aggregate and asphalt mastic were separated from the sliced image of HMA using features clustering method. Then the aggregate ranges were filled with balls as DEM “clump” using chain code boundary tracking algorithm and lines table filling algorithm. Finally, the asphalt mastic regions were filled with balls. The simulation test of dynamic deformation process of asphalt mixture under penetration load are in good agreement with those from practice.