For the main problems appearing in the process of reducing the number of either colors or gray level of the image, for example, the computation tends to be too complex and the image deviation tends to be too big, the method of image color quantization based on the clustering of neighborhood gray level is proposed. Firstly, the one-dimensional gray level expression of pixels is carried out by integrating the space and the gray information of the neighborhood pixels. Then the pixels are clustered by using the improved fuzzy ??-means clustering algorithm. The theoretical analysis and the experiment show that the proposed method can reduce the complexity of the quantization computation and maintain the overall level and the partial details of the image, and the image deviation becomes smaller after the quantization, which has the certain practical value to the imagery processing.