A target tracking algorithm of IoU network tracking with adaptive weighted characteristic responses(IT-AWCR) based on the IoU network is proposed in this paper. Firstly, a determination strategy for the target searching area is designed according to the velocity of a target, the outputs of block3 and block4 convolutional layers in ResNet50 are used as the shallow and deep feature representations of the target respectively by means of theoretical analysis. Then, with performance indexes of the accuracy of target location and the anti-interference ability of the filter model, the weights of the target deep and shallow feature responses are computed adaptively through the optimization algorithm. The rough target position and bounding box are obtained by the weighted fusion response, and multiple candidate bounding boxes are obtained by perturbation operation, which are input into the IoU modulation-prediction network to predict IoU values, taking the bounding box corresponding to the largest IoU as the final predicted target bounding box. Finally, the sample set is updated according to the relevant learning weights of the training samples and the similarities between this samples. Based on the sample set, the sparse optimization strategy is used to achieve the filter model update. The results of experiments on the OTB2015 and VOT2018 show the effectiveness of the proposed algorithm.