Histogram-based earth mover’s distance(EMD) (H-EMD) as an extension of EMD model is proposed for computing the cross-bin dissimilarity between histograms. For H-EMD model, the complexity of computing the cross-bin dissimilarity is reduced by eliminating the number of variables in linear program. On this basis, H-EMD is further adopted for computing the dissimilarity of shape context, and the shape matching method based on shape context descriptor is improved. H-EMD provides outperformance in computational complexity, and the experimental results show that the shape recognition can be effectively achieved by using the improved shape matching method.