Feature space analysis is the critical factor of visual tracking. Considering the issue of template matching within the mean shift framework, this paper proposes a concept of feature contribution, which can effectively reduce the influence of background feature and noise, and make importance feature play a key role. In addition, the binary distribution of structure introduced can effectively reduce the error of statistical features by spacial information and improve the tracking accuracy and robustness in a certain extent. Experimental results show that the proposed tracking algorithm is effective and also exhibits good result in the presence of noise, deformation and occlusion.