This paper considers the design of a privacy-preserving distributed Nash equilibrium seeking algorithm for aggregated games. Specifically, we address scenarios where there is no central node, and in such cases, each player cannot directly obtain the aggregated strategy information required for strategy updates. In this paper, a dynamic tracking consensus protocol is employed to estimate this information. The state variables used by players to estimate the aggregated strategies are regarded as sensitive information that requires protection. To safeguard the players' privacy, independent Gaussian noise is used to perturb the players' gradient information. By combining the Frank-Wolfe method with the dynamic tracking consensus protocol, we design a distributed Nash equilibrium seeking algorithm for constrained aggregated games under time-varying communication topologies. Further, we analyze the variance bounds necessary for the algorithm to achieve $(\epsilon,\delta)$-differential privacy. Furthermore, by analyzing the convergence of the estimation error of the aggregated term, we derive sufficient conditions for the convergence of the algorithm and provide a proof of convergence. Finally, the effectiveness and superior convergence speed of the proposed algorithm are validated through numerical simulations.