The multi-task particle swarm optimization(MTPSO) algorithm is widely used to solve multi-task multi-objective problems due to its rapid convergence via knowledge transfer learning. However, the MTPSO has strong randomness and is lack of guideness during search process, which is prone to fall into local optimum and has poor convergence performance. This paper proposes a Q-learning-based multi-task multi-objective particle swarm optimization algorithm(QM²PSO) via using learning and prediction of reinforcement learning to guide optimization. Firstly, we design the adaptive parameter adjustment method, which can update the inertia weight and acceleration parameters online based on Q-learning to improve the convergence ability. Secondly, we develop a mutation strategy based on Cauchy distribution, which can balance exploration and exploitation to avoid falling into local optimum. Finally, we design a knowledge transfer method based on the positive transfer criterion via updating the knowledge transfer rate based on \emphQ-learning to avoid negative knowledge transfer. The comparative results demonstrate that the QM²PSO is superior to the existing algorithms on convergence performance.