To improve the structural rigidity of multi-tower cable-stayed bridges, a novel crossed cable arrangement is proposed. By deriving and analyzing the restraint stiffness formula of the middle tower’s crossed cables, the optimal position for maximum restraining stiffness was investigated, leading to the proposal of an asymmetrical cable arrangement. Finite element models of three-tower and four-tower cable-stayed bridges were established, accounting for cable sag effects and large structural displacements. The influence of the asymmetrical cable arrangement on tower-beam deformation and tower forces was analyzed. Results show that when the height-to-span ratio of the multi-tower cable-stayed bridge ranges from 0.2 to 0.3, the optimal position for the crossed cables is approximately 0.7 to 0.76 times the span length form the middle tower. Compared to the traditional symmetrical arrangement, the asymmetrical configuration reduces the horizontal displacement at the top of the middle tower by 10.8% and 11.9%, and decreases the mid-span deflection under uniform load by 3.3% and 0.2% for the three-tower and four-tower cable-stayed bridges, respectively. Additionally, the first-order vertical bending frequency of the main girder increases by 3.5% and 6.4%, while the bottom bending moment of the middle tower decreases by 14.1% and 8.1%. These findings demonstrate that the asymmetrical arrangement significantly improves the restraining effect of crossed cables on the middle tower, increases overall structural rigidity, and improves the load-bearing performance of the middle tower.