The construction of smart parks promotes the development of enterprises and cities, and traditional park management methods are no longer suitable for smart parks with industrial integration and innovation. This paper takes Caojiatan Park as an example to design the overall framework of the smart park platform. Aiming at the problems of poor recognition environment, low efficiency and low accuracy in the park’s identity recognition, this paper proposes an identity recognition algorithm based on multi-modal and multi-core learning. The proposed algorithm divides the data in the video data into images and audio, and collects the text of personal information, and inputs the information of the three modalities into the same sample space. By introducing a multi-core learning algorithm with interval constraints, the difference is retained to the greatest extent. The difference and similarity of modalities are combined with feature fusion and decision fusion, and finally the classifier and scoring mechanism are used to output the identification results. Through experiments on the public video dataset and Caojiatan Park dataset, the experimental results show that the algorithm proposed in this paper has a maximum accuracy of 97.2%, which has a great advantage over traditional algorithms.