Aiming at the dynamic optimization of structure and parameters of the radial basis function(RBF) neural network, an optimization algorithm based on sensitivity analysis(SA) and particle swarm optimization(PSO) for the RBF neural network(SAPSO-RBF) is proposed. Firstly, the number of particle information is randomly initialized, and the particle information is added and deleted by the sensitivity analysis in the learning phase, and the network structure of the algorithm in first convergence is determined. Then, after the algorithm reaches convergence, we analyzes the sensitivity of the optimal particles, deletes the redundant information, and makes the algorithm re-divergent. An inertia weight update method is proposed to make the algorithm perform multiple divergence and convergence in the solution space, which enhances algorithm search ability while reducing network structure, and the convergence of SAPSO algorithm is proved. Finally, the results of experiments show that the proposed SAPSO-RBF algorithm has good self-organizing ability and has greatly improved the network structure compactness and accuracy compared with some other existing methods.