Considering the characteristics of assembly process such as multiple workers collaborating, the demand for workers with different skills, and mixed-model assembly, this paper proposes a Double Deep Q Network (DDQN) based algorithm to address Multi-manned Cooperation Mixed-model Assembly Line Balancing Problem. Firstly, a mathematical model for Multi-manned Cooperation Mixed-model Assembly Line Balancing Problem is established with the objectives of optimising the number of workstations and workers, the workload between workers and workstations. Secondly, the state space is designed based on the features of production objects. Meanwhile, the action space is designed using heuristic rules. Besides, the reward function is constructed based on the objectives of the model. As a result, the mathematical model is converted into a Markov decision process model. On this basis, an improved DDQN algorithm with an adaptive exploration probability for action decision-making and a decoding method based on worker utilization rate is developed. Finally, the improved DDQN algorithm is compared with the improved Discrete Water Wave Optimization algorithm and the Simulated Annealing algorithm on standard mixed-model assembly line test cases and multi-manned collaborative mixed-model assembly line test cases to verify the accuracy of the algorithm and the effectiveness of the model. The effectiveness and practicality of the algorithm are also verified by applying it to balance optimization in a practical car body mixed-flow assembly process.