A Galerkin Boundary Elements was applied to solve the first kind of integral equation with hyper-singularity, which can be deduced from the direct boundary integral formula for the Neumann problem of Laplace equation. The concept of integration by parts in the sense of distributions was used. When boundary rotation is introduced, the two order derivatives of singular kernel are shifted to the boundary rotation of unknown function in the Galerkin variational formulation. While linear boundary elements are used for 2-dimensional problems, the boundary rotation on each element can be discretized into a constant vector, so that the integration can be performed in a simple way and the difficulty of numerical calculation for hyper-singularity is overcome. The results of numerical examples demonstrate that the scheme presented is practical and effective.