Fault diagnosis of electric submersible pumps is crucial to ensure safe and reliable oil recovery, but the complex characteristics of electric submersible pump data, such as multivariate, nonlinear, and dynamic changes, pose a great challenge to this task. In recent years, the powerful capabilities of deep learning in feature extraction of complex data have spawned a series of neural network-based fault diagnosis methods for electric submersible pumps. However, most of the methods ignore the dynamic characteristics of the electric submersible pump data as well as the difficulty of long-term dependency feature extraction. To address the above issues, a multivariate time-series tokenized Transformer neural network is proposed for fault diagnosis of electric submersible pumps. This model designs a new multivariate time series tokenization strategy and inherits the advantages of the traditional Transformer neural network"s encoder which introduces multi-head attention mechanisms and residual connections in long-term dependency feature extraction. In addition, this model replaces the traditional Transformer neural network"s decoder with a forward neural network to simplify the model"s complexity. The effectiveness of the method is verified by analyzing the field"s faulty data. The experimental results show that the proposed method can accurately diagnose ten types of electric submersible pump faults, and it outperforms the popular deep learning methods.