An $H_\infty$ output tracking control problem is considered for a class of polynomial systems with partially unknown states under external disturbances. Firstly, an observer-based output tracking controller is presented from the perspective of the feedforward and feedback composite control method. The feedback stabilization controller is employed to ensure the stability of the closed-loop system, and the feedforward compensation controller is used to track the output signal of the reference model. Furthermore, a tracking controller is proposed with an output feedback structure, its advantage is that the observer and controller can be given separately, satisfying the separation principle and reducing the computational complexity. Then using a homogeneous polynomial Lyapunov function dependent on whole state variables, the sufficient condition of the asymptotic stability with $H_\infty$ tracking performance is derived for the closed-loop system. The corresponding observer and controller can be obtained directly by the polynomial sum of squares convex optimization technique. Finally, numerical simulation examples are given to verify the validity and superiority of the proposed method.