In the existing detection framework, the weak classification ability of the shallow layer is the key that restricts the further improvement of network accuracy. In order to solve the problem, a dynamic refinement detection network based on shallow positioning information is proposed. Based on single-stage algorithms, the network enhances the features of the shallow layer by adding multiple connection modules and removes the classification operations of the shallow layer to retain the location results of the shallow layer to the maximum. The location is used as the default boxes of the deep-level network. The deep level network is constructed by using a receptive field module with adaptive factors to obtain rich semantic information for the discrimination and fine-tuning of the regression results from the shallow layer. Finally, the designed deformable convolution head based on self-attention can automatically calibrate the position by shifting the detection box, which helps the network obtain accurate detection results. The experimental results on PASCAL VOC and MS COCO datasets show that the proposed network architecture achieves excellent detection accuracy.