In order to solve tracking failures caused by objects deformation, occlusion and fast motion, an algorithm called mean shift-tracking learning detection(MS-TLD) under the framework of the classical tracking-learning-detection(TLD) algorithm is proposed, which reconstructs a new tracker using the scale-adaptive mean-shift method. By introducing color histogram features and scale-adaption, the new tracker can track objects with deformation and fast moving. A new tracking-detection feedback strategy for the inter-correction between tracker and detector is designed, by which the proposed algorithm has better robustness when objects are occluded. The TB-50 standard dataset is used to verify and evaluate the proposed method. The experimental results show that the proposed algorithm can overcome the tracking failures caused by objects with deformation, occlusion, fast motion, as well as background clutters, and has better tracking accuracy and robustness compared with the TLD and other 3 classical algorithms.