An output feedback sliding mode control method without inverse model compensation is proposed to address the problem of the hysteresis between output displacement and input current affecting the control accuracy in a giant magnetostrictive actuator (GMA) system. Firstly, the classical Prandtl-Ishlinskii (CPI) model is improved, and a modified PI (MPI) hysteresis model is constructed to characterize the asymmetric hysteresis phenomenon. The MPI model is then decomposed into linear and nonlinear parts to convert the GMA system model into an affine form. Subsequently, an unknown system dynamics estimator (USDE) is designed to estimate the nonlinear part containing the hysteresis operator accurately. Finally, a sliding mode control method based on the fast-reaching law is introduced, which can improve the response rate and stability for a given trajectory. The experimental results show that for a given sinusoidal signal, the proposed method reduces the mean absolute error (MAE) by 25% and 23%, the root mean square error (RMSE) by 21% and 25%, and the standard deviation (SD) by 15% and 27%, respectively, compared to the ESO-based and CRRL-based control methods.