The axle box bearing of high-speed train bogies operates under highly uncertain and dynamic conditions, while the scarcity of effective fault samples leads to low diagnostic accuracy. To address this issue, this paper proposes an intelligent fault diagnosis method based on an MCWT-Li-ViT network. First, a discrete working condition identification method based on multi-dimensional statistical features is introduced. This method constructs a feature vector using key time-frequency domain features and uses a comprehensive stability scoring function to set adaptive thresholds, thereby improving the representativeness of state data and the stability of condition partitioning. Second, a continuous wavelet transform (CWT) knowledge base containing multiple mother wavelets is constructed to convert one-dimensional vibration signals into two-dimensional time–frequency representations with multi-perspective characteristics, enhancing the diversity and discriminability of fault features. Furthermore, a lightweight vision transformer (Li-ViT) is designed as the feature encoder, in which the multi-head attention mechanism is replaced with a broadcast attention mechanism to significantly reduce computational complexity while maintaining high recognition accuracy, thereby improving inference efficiency and deployment suitability on edge devices. Finally, a contrastive loss function based on multi-modal time–frequency features is designed to effectively train the MCWT-Li-ViT network, and a fault detection algorithm based on cosine similarity analysis is developed to accurately identify bearing health states. Validation on a dataset from a bearing test rig, using the Jetson Orin NX embedded platform as the core test environment, shows that the proposed method achieves an average fault detection accuracy of 97.2% under complex working conditions using only normal samples for training, demonstrating excellent engineering applicability and generalization capability.