In order to mitigate the problem of low efficiency and slow convergence of the multi-agent deep deterministic policy gradient(MADDPG) algorithm, the prioritized experience selection mechanism of MADDPG algorithm is studied and PES-MADDPG algorithm is proposed. Firstly, the model and the training method of the MADDPG algorithm are analyzed, the multi-agent experience buffer pool is ameliorated, and the priority evaluation function is designed based on the error of critic function and the training frequency of experience. The priority is treated as the selection probability to obtain the learning sample for training neural network. Finally, six groups of comparative experiments are conducted in both cooperative navigation and competitive environment. The experiments results show that the prioritized experience selection mechanism improves the training speed of the MADDPG algorithm, and the trained agents have better performance. The prioritized experience selection mechanism also has certain applicability to the training of multi-agents controlled by the deep detcrministic policy gradient(DDPG) algorithm.