Aiming at the problem of stable tracking control of quadrotor trajectory, a double-loop robust control system is designed to suppress the influence of external disturbance and parameter uncertainty. Firstly, the output regulator is designed for position control based on the dynamic surface internal model method, which can solve the problems of asymptotic tracking and disturbance rejection. Then, the higher order sliding mode is used to design the attitude controller, which can realize the global finite-time convergence, eliminate the chattering and relative order limitation of the system. In order to further improve the control precision, a robust accurate differentiator is adopted to accurately differentiate the attitude angle command signal. Finally, the strict mathematical proof for the stability of the system is given, and compared with PID control and traditional sliding mode control, the simulation results verify the superiority and robustness of the proposed control strategy.