The problem of fault diagnosis for the dynamic system is studied based on the data driven method of nonlinear spectrum. An extraction method of nonlinear frequency spectrum feature is proposed by using one dimensional nonlinear output frequency response function. In order to improve timeliness, the variable step size adaptive identification algorithm is used to solve the nonlinear output frequency response function. The step size is changed according to estimating error so that convergence rate and steady state error are both considered. After obtained nonlinear frequency spectrum feature, the least square support vector machine classifier is used to fault identification. The fault diagnosis of hoisting equipment is researched, and experiments show that the proposed algorithm has the good high recognition rate that can fulfill the demand of online diagnosis..