A multi-objective optimization model for the problem of gesture segmentation is proposed, and a method of solving the above model based on evolutionary algorithms is presented. When building the model, the positions of a series of pixels are taken as the decision variable, and the differences between the color of the pixels and that of a hand are taken as the objective functions. In addition, a multi-objective estimation of the distribution algorithm is presented based on the correlation among the positions of the hand pixels to solve the model so as to get the best pixel set, thus forming the human gesture. The experimental results show the effectiveness of the proposed model and algorithm.