This paper proposes an improved method based on YOLOv2 to solve the problems of low Chinese character detection rate, difficulty in small character detection and various character detection categories in natural scenes, and applies it to Chinese character detection in natural scenes. Firstly, k-means++ clustering algorithm is used to cluster the number and aspect ratio of character target candidate boxes (anchors). Then the multi-layer feature fusion strategy is proposed, the feature map output before the fourth maxpooling pooling layer in the original network is convolved with 3times3 and 1times1 convolution kernels and 4 times downsampling is performed to obtain local features, and the feature map output before the fifth maxpooling pooling layer in the original network is convolved with 3times3 and 1times1 convolution kernels and 2 times downsampling is performed to obtain local features. At the same time, repeat convolution layers in high-level convolution are added, and the number of continuous and repeated 3times3times1024 convolution layers in high-level convolution is increased from 3 to 5. Finally, the Chinese text in the wild (CTW) data set is used to compare the YOLOv2 algorithm with the improved one. The experimental results show that the improved YOLOv2 algorithm has a mean average precision (mean average precision, mAP) of 78.3% in Chinese character detection, which is 7.3% higher than mAP value of the original YOLOv2 algorithm, and is significantly higher than the one of other Chinese character detection methods in natural scenes.