Aiming at the high performance requirements of longitudinal control of a fixed-wing unmanned aerial vehicle(UAV), a performance optimization structure of the control system is proposed. This structure includes a nominal controller that stabilizes the system and an incremental controller that participates in performance optimization. The incremental implementation of the control system does not change the original control system, but compensates the control input and optimizes the control performance for the nominal control system exclusively. Based on the Q-learning theory, the incremental controller is designed. For the system with completely available state information, an incremental Q-learning algorithm based on state feedback is developed. When the state information cannot be obtained completely, an incremental Q-learning algorithm based on output feedback is designed by using the system input, output and reference trajectory data. Both incremental controllers learn incremental control laws adaptively in the data-driven environment without the need for system dynamics model and the control gain of the nominal controller. In addition, it is proved that the incremental Q-learning method has no bias in solving the Q-function Bellman equation under the excitation noise. Finally, the effectiveness of the method is verified by the simulation of an example of the longitudinal model of the F-16 aircraft.