To enhance the online prediction performance of evolutionary fuzzy systems in chaotic time series, an evolving fuzzy system based on kernel conjugate gradient(EFS-KCG) is proposed. The model explores and captures the fuzzy rules hidden in the time series through a structural evolution-based antecedent part. Meanwhile, the consequent part based on parameter update organically combines sparse learning strategy and kernel conjugate gradient algorithm to reduce the computational complexity and improve the convergence performance of the model. The EFS-KCG not only eliminates the redundant information in the samples, but also effectively balances the prediction accuracy and efficiency of the model. Experiment results show that the proposed model is effective in online prediction tasks for both benchmark and real chaotic time series.