For the problem that the data feature distribution of the same state in the source domain and target domain of rolling bearing is quite different under varying loads, and the model needs to be retrained for data update when the target domian data is obtained online in a sequential manner, a fault diagnosis method of rolling bearing under varying loads with the deep online transfer CNN-ISVM(convolutional neural networks-incremental support vector machine) is proposed. Short time Fourier transform is used to obtain the time-frequency spectrum of rolling bearing vibration signals under different loads, and the data sets can be constructed; the source domain data is used to build the CNN-ISVM pre-training model, and the model parameters are saved; transfer learning is used to transfer the source domain shared parameters to the target domain CNN-ISVM model training process to quickly establish the classification model; the ISVM classifier in the classification model retains the learned knowledge and processes the new data in the target domain online without being retrained. The experimental results verify that the proposed method can realize multi-state online classification of rolling bearings under varying loads with data collected in a sequential manner, and has better stability and higher accuracy.