Due to the wide visual angle range and complex environment of UAV visual tracking, UAV flight vibration, object occlusion, similar objects and other problems are often encountered, resulting in the drift of UAV tracking targets. Therefore, this paper improves the full convolution siamese network tracking algorithm (SiamRPN) with regression calculation, and proposes a UAV visual tracking algorithm (SiamDFT) to enhance the correlation of depth features. Firstly, the network width of three-layer convolution after full convolution neural network is doubled, and the feature extraction of template frame and detection frame is completed by making full use of the appearance information of the object. Then, the attention information fusion module and feature deep convolution module are proposed in the detection frame and template frame respectively. The feature correlation calculation methods of the two depths can effectively suppress the background information, enhance the correlation between pixel pairs, and efficiently complete the tasks of classification and regression. Finally, the depth cross-correlation operation is used to calculate the similarity, and the calculation method of distance intersection over union is introduced to locate the target. The experimental results show that the accuracy and success rate of SiamDFT are 79.8% and 58.3% respectively in UAV short-term tracking scene, and 73.4% and 55.2% respectively in UAV long-term tracking scene. Finally, the real-world test is completed to fully verify the effectiveness of the algorithm.