Distributed data stream generated by current data-driven applications has become a main data representation. Although distributed data stream is captured from different data sources, they are correlated to a common event. Hence, the key issue of distributed online learning is how to build global learners by sharing data of local node. For this problem, this paper proposes a sharing data solution for distributed online learning, containing the semi-supervised clustering approach based on exponential loss and the sharing data approach based on covariance matrixes and mean vectors, and proves the cumulative absolute error between the rebuilding data set and the original data set is bounded on the given threshold under some probability. Experimental study demonstrates that the proposed approach has lower network traffic between nodes, and gets the learner having better generalization capability.