The author analyzs computational complexity of previous MPEG-4 video codec system in terms of data processing, memory accessing and address computation, and optimizs the performance of data processing and data transfer of MPEG-4 video coding and decoding algorithm. The architecture model of the real-time MPEG-4 video codec is proposed based on the analyses. This architecture can effectively utilize computational resource in software and improve the performance of MPEG-4 video codec system.