If measurements contain a high proportion of outlier data, the numerical stability and robustness of existing robust Kalman filters may be seriously degraded. Therefore, this paper proposes a novel robust Kalman filter based on the approximate least absolute deviation and modified Gauss-Newton method to reduce the adverse effects of the outlier measurement on the filtering. Through the analysis based on the condition number and influence function, it is proved theoretically that the numerical stability and robustness of the proposed approach are preferable to those of Huber-based Kalman filter. The theoretical analysis results are verified by the simulation experiments. The simulation results show that the performance of the proposed filter is almost equivalent to that of the Huber-based Kalman filter in the case with low proportion of outlier measurements. However, the proposed filter exhibits more superior performance than the Huber-based Kalman filter in the case with high proportion of outlier measurements. The comparison of computational cost of several filters is also performed by the simulation experiments. It is found that the computational cost of the proposed filter is less than the computational cost of the Huber-based Kalman filter.