For assembly sequence planning problems, most of the existing algorithms focus on a single target configuration. For multi-target configurations and large-scale problems, existing algorithms often have dimension disaster problems with poor generalization ability. Therefore, this paper uses the characteristics of the hierarchical structure of assembly sequence planning problems and conducts a general assembly sequence planning method based on hierarchical reinforcement learning, which is suitable for multi-configuration assembly tasks. First of all, this paper constructs the assembly sequence planning problem as a hierarchical Markov decision process, in which the upper layer performs sequence planning, and the lower layer carries out workpiece motion planning, which conforms to the hierarchical structure of the assembly process, making the planning method more flexible and interpretable. Then, in view of the hierarchical Markov decision process, this paper proposes a general assembly sequence planning algorithm based on hierarchical reinforcement learning, which improves the adaptability and generalization ability of the planning method to multi-target tasks and the information utilization of the target configuration. Finally, the proposed method is verified on the built simulation platform. The results show that the proposed method can extract general information about assembly problems, and it has good decision-making ability for any initial state and other various configurations assembly tasks, which verifies the effectiveness and flexibility of the method. Thus, a more general and flexible assembly sequence planning algorithm suitable for multiple configurations is realized.