The optimal safety tracking control problem with prescribed performance is investigated for a class of nonlinear systems with unknown initial tracking condition. A new method for performance constraint control design is developed based on a variable barrier function. Based on the existing secure boundary protection method (SBPM), a novel secure boundary self-adjustment law (SBSAL) is proposed. It can not only handle the situations that the actual output constraints suddenly change, but also solve the problem that the system output is not able to quickly and accurately track the original expected trajectory after the mutation is relieved, so that the safety tracking control strategy is more consummate. Meanwhile, the reinforcement learning (RL) optimal method based on actor-critic neural networks (ACNNs) is adopted to optimize the control input of the system, and reduce the energy consumption for control. The designed optimal safety tracking controller with prescribed performance constraint can ensure the safe tracking control of the system with unknown initial tracking condition, and the output of the system has prescribed finite-time control performance. Finally, the effectiveness of the proposed method is verified by simulations.