With the wide application of object tracking technology in a variety of vision tasks, the real-time requirements for tracking algorithms are becoming increasingly important. Although the fully-convolutional siamese network algorithm for object tracking(SiamFC) is ideal in tracking speed, it is prone to tracking drift when dealing with complex tracking environment. In order to simultaneously improve the tracking accuracy and speed, a high-speed tracking algorithm based on negative example mining and feature fusion is proposed. Firstly, the improved lightweight network ShuffleNetV2 with the added crop layer is used for feature extraction, which can learn deeper features without adding extra parameters and calculations. Then, different types of negative example pairs are introduced in the offline training phase to strengthen the learning of semantic information, aiming at improving the feature discrimination ability of the model. Finally, a multi-scale feature fusion strategy is adopted, which makes full use of shallow and deep features to obtain a higher quality response map and greatly improve tracking accuracy. The experiment results of OTB100 and VOT2018 datasets show that the proposed algorithm significantly outperforms the benchmark algorithm SiamFC in various indicators, yielding 8.3% gain in the OTB100 dataset and 7.9% gain in the VOT2018 dataset. At the same time, the proposed tracker can perform 114FPS under NIVIDA GTX 1070.