A counterfactual progressive identification framework was developed to distinguish improvements under normal operations from additional net resilience gains caused by increases in connected and automated vehicle (CAV) penetration under signal control failure conditions. An interaction scenario set covering 11 CAV penetration levels and 4 failure intensity levels was established in SUMO. A macroscopic fundamental diagram, a common random numbers (CRN)-based paired design, static difference-in-differences (DID), event-study dynamic DID, and stage-decomposed difference-in-difference-in-differences (DDD) were combined to quantify the overall net effect, its temporal evolution, and the phase-specific mechanisms. Block bootstrap resampling was further used for statistical inference. Results showed that the reduction in average vehicle time loss under the no-failure scenario increased from 0.03 s/veh to 0.79 s/veh. After the baseline effect under normal operations was removed, additional net resilience gains became stable at penetration rates of about 40% under L2, 50% under L3, and 60% under L1. At full penetration, the overall gains under L1, L2, and L3 reached 1.00, 1.15, and 1.29 s/veh, respectively. Marginal net gains were concentrated mainly in the 50% to 70% interval, and the peak value under L3 reached 0.14 s/veh. Net gains were generated mainly in the late failure stage and the recovery stage, and the recovery-stage net effect was markedly stronger than the failure-stage net effect. The results indicate that the primary role of increasing CAV penetration is to strengthen system recovery and reduce cumulative losses.