Because of the strong nonlinearity and model uncertainty in the integrated navigation system, the classical extended Kalman filter cannot satisfy the actual application requirement of the integrated navigation system. The concrete analysis of the estimation performance of the Gaussian nonlinear filter under Bayes framework is given. Firstly, the Taylor approximation is obtained by the Taylor expansion of the nonlinear function at the estimation points, and the approximate precision of the filter algorithm is analyzed by the first and second moment. Then, the nonlinear filter algorithm is analyzed by the numerical stability. Finally, the low-dimensional and the high-dimensional test model is used to analyze and compare several Gaussian filter algorithms. The results provide reference for the practice of the integrated navigation system.