How to realize the coordinated development between economic development and environmental quality has become a hot issue. Based on the data from 1997 to 2016 in China, this paper empirically studies the influence mechanism of fiscal decentralization and economic agglomeration on the environmental pollution. The empirical results show that fiscal decentralization and economic agglomeration have negative and positive effects on environmental pollution. Economic agglomeration can promote the reducing effect of fiscal decentralization on environmental pollution. Further analysis is made on the region level, and it is found that the differences in economic base and resource endowment will lead to different mechanism of effect of fiscal decentralization and economic agglomeration on environmental pollution. Fiscal decentralization leads to aggravation of pollution in the northeast and eastern regions, and the reduction of pollution in the central and western regions. The economic agglomeration is obviously beneficial to the financial decentralization's reducing pollution in eastern and western regions. The reform of fiscal decentralization and the assessment mechanism of local governments should be promoted according to local conditions so as to improve the quality of the environment.