Complex industrial processes are often subject to fast and slow coupling dynamic characteristics. Using the traditional cascade design approach cannot ensure the optimal operation performance of the whole system, while the existing integrated design methods tend to result in high dimensionality and ill-conditioned numerics. For a class of complex industrial processes where both unit device and operational process are unknown, we propose a parallel fast and slow reinforcement learning-based composite non-cascade control approach using singular perturbations. Firstly, the optimal operational control problem of complex industrial processes is modeled as a non-cascade optimal control problem of a two-time-scale system introducing a convergence factor. Secondly, the original optimal control problem is decomposed into an optimal regulation problem of a fast subsystem and an optimal set-point tracking problem of a slow subsystem via singular perturbation theory. Thirdly, we design a data-driven iterative algorithm to learn the optimal controllers of both fast and slow subsystems in the framework of reinforcement learning, and then we construct a composite optimal control strategy that is independent of model parameters. Compared to the existing methods, the proposed approach does not require knowledge of model parameters of device layer and achieves zero-error asymptotic tracking of desired operational index. Finally, we validate the effectiveness and superiority of the proposed method through experimental results conducted on mixed separation thickening process.