With large-scale integration of distributed energy resources, operational uncertainty in distribution networks has increased significantly, making dynamic distribution network reconfiguration (DDNR) essential for secure and economical operation. Deep reinforcement learning (DRL) is applied to the DDNR due to its adaptability to uncertainty and data-driven nature. To address the curse of dimensionality and action blind zones in discrete-action DRL-based reconfiguration, this paper proposes a phased structure-aware (SA) DRL framework. A performance- and structure-driven topology screening strategy is first developed to construct a compact and high-quality action space. Then, an SA-Q network integrating graph convolutional networks and action embedding is designed to encode topologies into graph embeddings, enabling structural awareness and improved generalization. The model is incorporated into an enhanced Rainbow DQN framework for offline training and online decision-making. Simulation results validate that the proposed method effectively reduces network losses, improves voltage profiles, and minimizes unnecessary switching actions, achieving fast and stable convergence while significantly enhancing distribution network operational performance.