Facial expression recognition is still faced with great challenges due to the intra class variation and interclass interference. In this paper, a deep facial expression recognition based on random forest with gender constraints is proposed to solve the problems of noise, gender and other variation and interference. Firstly, robust facial features are extracted by deep multi-instance learning to solve the problems of illumination, occlusion and low resolution. Secondly, the face expression classifier is designed by using gender conditional random forest classification to solve the problem of gender interference. Extensive experiments on the public CK+, BU-3DEF and LFW databases show that the recognition rate of our method is 98.83%, 90% and 60.58% respectively, which has better performance and robustness compared with the state-of-the-art methods. In addition, compared with other advanced deep learning methods (requiring a large number of training databases), our method only needs a small number of training samples to achieve better results.