Using multi-agent reinforcement learning methods to enable heterogeneous multi-entity systems is a frontier issue in the field of distributed artificial intelligence. Efficient coordination among heterogeneous multi-formation is the key to winning in the task of multi-aircraft collaborative penetration of sea targets. The exploration efficiency of multi-agent reinforcement learning is not high because of the partial observability of the environment. Therefore, this paper proposes a knowledge-based hierarchical value decompostion(HAVED) strategy planning method for multi-aircraft coordinated penetration operations. The upper layer starts resource scheduling around intra-team occupying planning. The lower level is organized around inner-team task planning. The value decomposition base algorithm uses the weighting operator to weight the loss corresponding to the joint action to avoid falling into the local optimal, so as to improve the exploration and learning efficiency of multi-aircraft and multi-formation penetration strategies in confrontation scenarios. In order to verify the effectiveness of the proposed algorithm, the typical task scenario of multi-aircraft collaborative penetration of sea targets is designed. By using the centralized training-distributed execution paradigm, simulation experiments are carried out on the Mozi Wargame inference platform, and the multi-class value decomposition methods are compared and analyzed, and the effectiveness of the proposed method is verified. Finally, by analyzing the process data of confrontation wargaming, three typical action strategies emerged by the agents are summarized.