To enable collaborative processing of heterogeneous industrial tasks in the scenario with multiple devices and multiple edge servers, this paper proposes a multi-task end-edge offloading algorithm based on Lyapunov optimization and deep reinforcement learning. First, to jointly optimize task offloading decision, offloading ratio and transmit power, a long-term average system overhead minimization problem is formulated with full consideration of computing frequency, transmission power, long-term energy consumption and task deadline. As variables are coupled among different time slots in the long-term objective and constraints, the problem is difficult to solve. Thus, the long-term average system overhead minimization problem is decoupled into some independent time-slot optimization problems based on the Lyapunov optimization theory. By Markov decision process modelling and employing a double dueling deep neural network architecture, a deep reinforcement learning-based multi-task offloading algorithm is proposed. Experiments show that the proposed algorithm can converge stably, and can effectively reduce the long-term average system overhead under long-term energy consumption constraints and task deadline requirements.