After analyzing the entropy of the image, the conception of image multi-scale entropy is introduced, and image multi-scale entropy and the vector of image multi-scale entropy are defined. A region matching tracking method-infrared image matching tracking algorithm based on image multi-scale entropy is present. Firstly, the image multi-scale entropy and the vector of the image multi-scale entropy are calculated. Then the absolute distance(AD) of the vector of image multi-scale entropy is applied to matching and tracking. The experimental results show that the proposed algorithm has the robustness matching tracking properties and is robust to the problems of the rotation and geometry distortion of targets.