For the heterogeneous AGV flexible job shop scheduling problem considering energy constraints, this paper establishes a mathematical model with the objectives of minimizing the maximum completion time and total energy consumption. A hybrid optimization algorithm integrating Double DQN, namely DDNSGA-II, is proposed to solve the problem. First, according to the characteristics of the problem, a three-stage encoding scheme is constructed, and four hybrid initialization strategies are designed to significantly improve the quality and diversity of the initial population. Second, Double DQN is incorporated into the genetic algorithm framework, where the state space, action set, and reward function are designed to achieve adaptive adjustment of genetic operator parameters, thereby enhancing the solution efficiency and stability of the algorithm. Meanwhile, three neighborhood search strategies are developed based on the optimization objectives to improve the local search capability and convergence performance. Finally, ablation experiments are conducted to verify the effectiveness of Double DQN in dynamically adjusting genetic operator parameters and the proposed neighborhood search strategies. Comparative experiments with other optimization algorithms further demonstrate the superiority of DDNSGA-II in solving the FJSP-HA problem.