This paper presents a nonlinear control law based on the combination of reinforcement learning(RL) and super twisting methodology for the attitude control of a small-size unmanned helicopter, which is subjected to modeling uncertainties and unknown external disturbances. The proposed control law only uses input and output data of the helicopter to train the actor-critic(AC) neural networks to compensate for modeling uncertainties. Then a nonlinear robust controller based on super twisting methodology is developed to compensate for the unknown external disturbances. The Lyapunov based stability analysis is used to prove that the attitude error of a unmanned helicopter can converge to zero in finite time. Finally, the proposed control law is verified on a self-built hardware in the loop testbed. The experimental results show that the proposed control law can achieve good control performance together with good robustness for modeling uncertainties and wind disturbances.