A global motion estimation algorithm based on motion classification is presented. Firstly, the Harris feature points are selected evenly and matched by using feature window. Hence, the statistic features of all motions are analyzed according to different motion kinds including translation, rotation and zoom. Then, the fast motion classification method is proposed to validate all points. Thirdly, the remained global feature points are brought to the affine model to compute global motion. Finally, the Kalman filter is used to compensate each current frame. Experimental results show that the algorithm can correctly detect global motion in dynamic scenes with camera scan and various dithering. The estimation error is below 1/2 pixel at real-time stabilization, which can greatly improve the stability and fidelity of videos.