Using multi-agent reinforcement learning method to enable heterogeneous multi-entity systems is a frontier issue in the field of distributed artificial intelligence. Efficient coordination among heterogeneous multi-formation is the key to victory in the task of multi-aircraft collaborative attack on sea targets. Due to the considerable part of the environment, the exploration efficiency of multi-agent reinforcement learning method is not high. In this paper, a knowledge-based HierArchical ValuE Decompostion (HAVED) strategy planning method for multi-aircraft collaborative attack operations is proposed. The upper layer focuses on multi-aircraft formation occupying planning for resource scheduling, and the lower layer focuses on task planning for target allocation. The value decomposition base class algorithm uses weighting operators to weight the loss corresponding to the joint action, avoids falling into the local optimal, and strives to improve the exploration and learning efficiency of multi-machine multi-formation penetration strategy in anti-scene. In order to verify the effectiveness of the algorithm, the typical task scenario of multi-aircraft collaborative penetration of sea targets is designed. Using the centralized training-distributed execution paradigm, simulation experiments are carried out in the Mozi military chess simulation platform, and the effectiveness of the proposed method is verified. The process of confrontation is analyzed and three typical action strategies are summarized. The address of the project: https://gitee.com/jrluo2049/haved.