Classical Takagi-Sugeno-Kang(TSK) fuzzy systems take too much training time on the large scale datasets. To overcome this difficulty, the new TSK fuzzy system in probability modeling framework, i.e., Baysian L2-Norm-Takagi-Sugeno-Kang fuzzy system(B-TSK-FS), is proposed. The B-TSK-FS has the ability to handle large datasets. Specifically, based on the probabilistic output error of the L2-Norm TSK-FS, the B-TSK-FS has better generalization abilities through taking both antecedents and consequents of fuzzy rules into consideration simultaneously. In addition, the Dirichlet prior distribution is introduced to the B-TSK-FS, which makes the fuzzy memberships be represented sparsely, so as to condense the large scale datasets for reducing the running time. The experimental results on the synthetic and real-world datasets show the advantage of the proposed method.