The classical Coulomb and Rankine earth pressure theories are based on the assumption that the soils near a retaining wall are semi-infinite. It is obviously that these theories are no longer suitable for the narrow backfill behind a retaining wall. A series of numerical simulation are conducted with DEM to study the active failure process of limited cohesionless soil behind a rough or smooth wall. The development of the failure surface, the displacement field of the soils and the distribution of active earth pressure are analyzed. The results obtained from DEM show that multiple slip surfaces occur in the backfill if the soil behind the smooth wall is narrow enough. The slip surface angle is close to the Coulomb’s theoretical solution, which shows no related to the width height ratio. This ratio also has little influence on the active earth pressure in this case. In another case, when the wall is rough, the actual shape of failure surface is a curve rather than a straight line. The rougher the wall, the weaker the reflection. The angle of failure surface decreases along with the width height ratio of soils increases. And the failure surface is finally located inside the Coulomb’s failure surface. There exists a critical width height ratio of soils behind the rough wall. The active earth pressure decreases with the increases of the ratio if it is smaller than the critical ratio, but the active earth pressure is independent with the ratio if it is larger than the critical ratio. The smaller the width height ratio is, the smaller displacement of the ultimate equilibrium state will be, regardless of roughness of the retaining wall.