A novel improved mean shift algorithm based on texture and color features is proposed. Firstly, the moving object histograms of the color feature and texture feature are got respectively. Secondly, the fusion similarity coefficients are defined to fuse the different feature space, and the central location of moving object is calculated based on MS tracking. Thirdly, according to the color feature and texture feature, the object location is updated by using the feature adaptive coefficients. Finally, the above results are processed to get the final object location. Experimental results show that the proposed tracking algorithm exhibits good results in the presence of noise, deformation and occlusion.