When the attributes are ordered, the K-dependence Bayesian(KDB) classifiers only consider direct dependence between the classes and the decision attributes, but neglect the conditional correlation between them with the decision attribute as a condition. To fully express the dependency relationship between the classes and attributes and improve the expression of decision attributes for classification, we use conditional relationship between the classes as and the decision attributes to find an optimal attribute order based on the KDB structure. Besides, attribute selection is used to remove redundant attributes and prevent over-fitting. The structure learning and attribute selection are carried out based on the greedy search strategy. The proposed algorithm is tested on several UCI datasets and compared with other classical methods. The results show that the proposed approach obtains higher classification accuracy and outstanding robustness.