This paper addresses the prescribed-time time-varying formation tracking (TVFT) problem for heterogeneous multi-agent systems (HMASs) with bounded control gains under a directed graph. First, a prescribed-time switching observer is proposed for each agent to estimate the leader’s state. The prescribed-time switching observer switches from one observer with a time-based generator (TBG) term to another observer with a fractional-order term. Building on the prescribed-time switching observer, a formation tracking switching controller is presented and it switches from one TBG-based feedback controller to another controller with fractional-order feedback. The switching instants for the proposed prescribed-time switching observer and formation tracking switching controller can be arbitrarily pre-specified by a user. Then, using the Lyapunov stability theory, it is proven that the proposed controller can drive the formation tracking error to an adjustable domain at a prescribed switching instant and to zero within another prescribed time behind the prescribed switching instant. In contrast to the existing controllers with time-varying scaling functions, the proposed controller avoids the numerical realization problem caused by unbounded control gains. Finally, the effectiveness of the theoretical results are demonstrated through two simulation examples.