This paper considers the unknown underwater channel and the uncertain model parameters, and hence, a multiple autonomous underwater vehicles(AUVs) formation control algorithm based on deep reinforcement learning is proposed. Firstly, a least square estimator based on environmental sampling data is developed to predict the unknown channel in fading environment. Then, according to the sigal-noise ratio(SNR) obtained by the channel prediction estimator, the co-optimization problem of communication effectiveness and formation stability is established. Based on this, the formation control algorithm based on the depth deterministic strategy gradient algorithm(DDPG) is designed. Finally, simulation and experimental results verify the effectiveness of the proposed algorithm. According to the simulation results, compared with the direct formation control, the communication performance is improved by 13.5$%$ considering the communication efficiency.