The optimal safe tracking control problem with prescribed performance is investigated for a class of nonlinear systems with unknown initial tracking condition. A new method for performance constraint control design is developed based on a variable barrier function. Aiming to the safe tracking problem of systems caused by the sudden changes of the actual output constraints and the release of the sudden changes, a novel secure boundary self-adjustment law (SBSAL) is proposed, the existing secure boundary protection method (SBPM) is further updated. Meanwhile, the reinforcement learning (RL) optimal method with actor-critic (AC) architecture is used to optimize the control input of system. Based on this, a safe tracking controller with prescribed performance constraint is designed for the system. The controller can ensure the safe tracking control and prescribed finite-time control performance of the system with unknown initial tracking condition, and solve the problem that the output after the sudden changes are relieved fast tracks the original desired trajectory. Finally, the effectiveness of this proposed method is verified by simulations.