The image-based visual servo control method of robots obtains the image information through the robot's vision and then forms the closed-loop feedback based on the image information to control the robot's reasonable movement. However, due to the problem of poor robustness and slow convergence, the selection of servo gain for classical visual servoing is artificial assignment under most conditions. Therefore, an intelligent servo control method based on Dyna-Q learning is proposed to adjust the servo gain to improve its adaptability. Firstly, this method uses the image feature extraction algorithm based on Felman chain code to extract the target feature point, then uses the image-based visual servoing to form the closed-loop control of the characteristic error. Then, this paper presents a decoupling visual servoing control model for the dynamic characteristics of rotor UAV's strong coupling underactuated. Finally, a reinforcement learning model using Dyna-Q learning to adjust the servo gain is established, through which the rotor UAV can choose the servo gain independently. The Dyna-Q learning method learns to store experience on the basis of classical Q-Learning by setting up an environment model, and the virtual samples generated by the environment model can be used as learning samples to iterate the value function. The experimental results show that the proposed method is faster and more stable than the classical PID control and classical image based visual servo methods.