For RGB images instance segmentation, some segmentation errors may occur in areas with similar textures but different categories. This paper introduces depth information and makes use of three-dimensional geometric features of RGB-D images, proposing the double pyramid feature fusion model. The method constructs two pyramid depth networks with different complexity to extract RGB and Depth features of different resolutions, then add two features of corresponding resolution. In this way, we change the input features of region Proposal network, then the classification network, regression network and mask network output positioning and classification results to get RGB-D images instance segmentation results. The experimental results show that the proposed model can learn the complementary information between depth images and RGB images, and get satisfactory RGB-D instance segmentation results. Compared to the mask R-CNN model that does not contain depth information, the average precision of the proposed model is increased by 7.4%.