Traditional ensemble Takagi-Sugeno-Kang(TSK) fuzzy subclassifiers face such challenges, hierarchical learning have no interpretability, because of the presence of intermediate variables, and when a new TSK fuzzy subclassifier is added to or removed from the structure of the current fuzzy classifier, boosting learning must retrain each TSK fuzzy subclassifier by appropriately assigning new weights. Therefore, An ensemble framework EP-Q-TSK of TSK fuzzy subclassifiers with parallel learning way is proposed. The proposed framework has the following distinctive characteristics: 1) Each TSK fuzzy subclassifier can be built quickly with least learning machine (LLM) in parallel; 2) As a novel ensemble learning, the proposed framework augments the original validation data space with the outputs of each TSK subclassifier in an incremental and inexpensive way, and then speed up the final classification on the validation data by using the FCM and the KNN method; 3) Enhanced classification performance by FCM & KNN is experimentally revealed, and the experimental results on benchmark datasets indicate the effectiveness of EP-Q-TSK and its parallel learning method in the sense of both enhanced classification performance and interpretability.