Aiming at the problems that the current YOLOv5 algorithm detects steel surface defects with low accuracy and slow speed, a YOLO-Steel steel surface defect detection algorithm is proposed. First, a light-weight channel attention module is proposed, which can effectively focus on important channels with only a small computational cost. Secondly, by using atrous convolution to expand the receptive field, a light-weight spatial attention module is proposed. Finally, a pyramid attention structure is proposed, which uses multi-level pooling to scale feature maps, and uses spatial attention modules on feature maps of different resolutions to learn its spatial dependence information. After splicing in dimensions, the channel attention module is used to reconstruct its channel-related information, which can achieve better detection results for multi-scale detection targets. The experimental results show that the average mean precision(mAP) of YOLO-Steel on the steel surface defect data set can reach 77.2%, which is 1.8 percentage points higher than that of the YOLOv5s algorithm, and the model time and space complexity are basically the same as those of YOLOv5s. On the basis of ensuring the detection speed, the accuracy is effectively improved.