For the swing arm control of high precision spaceborne spectrograph, an integral sliding mode variable structure control(ISMVSC) is presented to deal with the disturbance problems of micro-gravity and micro-vibration. Firstly, the interference model of micro-gravity and micro-vibration on the swing arm is given. Then, according to the stability requirements of the optical path difference velocity(OPDV), the control method for the swing arm is designed based on the nonlinear relationship between the motor’s angular velocity and its OPDV. Finally, the performance of disturbance suppression for the swing arm system is simulated. Compared with the conventional PI control method, the simulation results show that the proposed ISMVSC is effective for guaranteeing the system’s robustness and inhibiting the disturbance introduced by the micro-gravity and micro-vibration in space.