Physics-Constrained and Gradient-Guided Reinforcement Learning for Secure Energy Dispatch in Microgrids
Author:
Affiliation:
Fund Project:
The National Natural Science Foundation of China under Grants No. 62373328;Zhejiang Provincial Natural Science Foundation of China under Grant No. LR25F030003.
Current microgrid energy scheduling faces critical challenges, where temporal coupling constraints lead to a significant expansion of the decision space, and AC power flow equations introduce nonlinear constraints that increase computational complexity, resulting in an overall optimization model with strong non-convexity and substantially increased solving difficulty.To address these issues, this paper proposes a microgrid energy scheduling method based on safe reinforcement learning and physics-constrained gradient guidance. The method constructs a deep learning-based action correction safety layer that projects agent actions into the feasible domain during environment interaction, ensuring operational physical feasibility while effectively improving exploration efficiency.Furthermore, by embedding this safety layer into the network training process, it enhances the $Q$-value estimation accuracy of the Critic network and improves the physical constraint learning efficiency of the Actor network. Experimental results on an IEEE 14-bus model-based microgrid with electro-hydrogen coupled power flow demonstrate that the proposed method outperforms the Lagrangian multiplier method (TD3-Lag) and the penalty-based method (TD3-Pen) in scheduling decision performance. Compared to the numerically optimized safety layer approach, it achieves approximately three orders of magnitude faster deployment speed while maintaining similar performance levels.