Due to the limited visibility of unmanned surface vessel(USVs) in multi-USV formation systems, this paper proposes a model with an unmanned aerial vehicle(UAV) as the leader of the USV formations. A fixed-time optimal formation control algorithm is designed for the formation of multi-USVs under UAV leadership. Building upon the three-degree-of-freedom dynamic modeling of USVs and UAVs, an augmented system is constructed. A performance index function that considers input cost and formation error is established. In the proposed reinforcement learning algorithm with an identifier-actor-critic structure, unknown nonlinear system dynamic is approximated by the identifier. The actor and critic for executing control behavior and evaluating control performance tunes simultaneously towards the optimal solution. Furthermore, it is proven that the formation error of the UAV-USV system can converge to a small region around zero within a fixed time. The effectiveness of the proposed algorithm is verified by simulation examples.