The technology of human abnormal behavior recognition and detection has been widely applied in various fields. Due to the problems such as object occlusion, illumination and visual angle changes and complex background in video, lightweight human skeleton becomes a competitive tool for processing such real-time tasks. Most researches have reviewed the methods relevant to this task from different perspectives, but there is a lack of work on human skeleton. Based on skeleton data, this paper systematically reviews the methods of human abnormal behavior recognition and detection under the background of deep learning. Firstly, according to the different number of targets in the application scenario, human pose estimation algorithms are classified and summarized. Secondly, based on the different feature extraction networks, the abnormal behavior recognition methods are divided into five categories, which are compared and analyzed around the CNN, RNN, GCN, Transformer and hybrid models. Then, from the perspective of data and label mapping learning, three types of abnormal behavior detection methods are discussed. Finally, the baseline datasets and the performance of related algorithms are introduced, and the challenges and prospects facing this task are discussed in order to provide reference for future research in this field.