The recurrent neural network(RNN) model combined with the attention mechanism is the current mainstream abstractive text summarization method, which uses a sequence-to-sequence framework based on deep learning. However, the abstractive summarization model based on the RNN has insufficient parallel ability or performance defects of long-term dependence, and the problem of low accuracy and high repetition rate in the process of generating summary. In order to overcome these problems, an abstractive summarization model method combining the BERT pre-training model and the convolutional gating unit is proposed based on the improved Transformer model. In the encoder stage, it makes full use of the large-scale corpus pre-trained by the BERT to replace the RNN to extract the contextual representation of the text, and then combines the convolutional gating unit to filter the output of the encoder to filter out the source text. In the decoder stage, three different Transformers are designed, for exploring a more effective fusion method of the BERT pre-training model and convolutional gating unit to improve the performance of text summarization. The ROUGE value is used as the evaluation index in the experiments. The experimental results on the LCSTS Chinese dataset and CNN/Daily Mail dataset show that the proposed method improves the accuracy and readability of the abstract.