A two-phase clustering algorithm based on the improved manifold distance as the similarity measure combined with the bee colony algorithm is proposed. Firstly, based on local density, max-min distance and neighbors selecting, data set is initialized, and the representative points are obtained. Then, the clustering algorithm is viewed as an optimization problem, in which the correctly category is obtained by getting the optimal clustering center through the improved bee colony algorithm dealing with the representative and unclassified points, and obtaining the overall consistency information of the manifold distance. Finally, the two phase algorithms are classified. Experiment results show that the proposed algorithm has better clustering results.