The shepherd control method is gradually being applied to address large-scale collective motion coordination problems, such as bird dispersal at airports, drone herding, as well as air-ground coordinated surveillance and guidance. Taking UAV herding as an example, a self-adaptive shepherding control method based on a deep $Q $-network(DQN) and hierarchical autonomous decision-making is proposed. Firstly, considering the factors such as the decay of the activity of outlying individuals, a perception and motion model of the shepherding control problem is established. Then, a global center of mass arc(GCM-Arc) control method and an obstacle avoidance strategy are proposed to improve the percentage of controlled individuals in the flock for the individual stagnation and outlier problem. Finally, a hierarchical autonomous decision-making model is established, and a hierarchical GCM-Arc control method is proposed by combining the GCM-Arc control method and the DQN, which realizes adaptive switching of control mode and adaptive adjustment of parameters. Simulation experiments demonstrate that the proposed method outperforms classical GCM-V(V-shaped trajectory based on global center of mass) and Arc-Formation shepherding control methods significantly in terms of shepherding task completion time, total drone distance traveled, average radius of the sheep herd, individual outlier rate, and sheep herding task success rate.