In view of the problems such as spectrum shortage and low utilization efficiency of wireless communication environment of smart grid, this paper applies cognitive radio technology to the neighborhood network communication of smart grid. The concept of cognitive smart grid is introduced to ensure the fairness and effectiveness of business transmission, and a spectrum allocation strategy based on the improved binary butterfly optimization algorithm(BOA) is proposed, which takes into account the signal-to-noise ratio and path loss in the communication process, and selects the system energy efficiency as the channel benefit. The modeling and simulation are carried out in urban residential areas with fixed topology. Firstly, spectrum are allocated for cognitive smart grid users with the binary butterfly optimization algorithm based on the improved time-varying conversion function and disturbance strategy(IBBOA). Then, the transmission power of the user is dynamically adjusted by the closed-loop power control algorithm based on the received signal-to-noise ratio, which reduces the interference between the cognitive smart grid users and the main users. Finally, with the system energy efficiency and two fairness indexes of users as the optimization objectives, the experiment of contrasting the proposed algorithm with the genetic algorithm(GA) and the binary particle swarm optimization(BPSO) algorithm is carried out. The simulation experiment shows that the IBBOA with joint closed-loop power control obtains system energy efficiency 33.2% higher than the NBOA without joint closed-loop power control, and the final system energy efficiency and the user fairness index fair of the IBBOA are 47.8% and 62.6% higher than the worst-performing GA respectively, and 17.6% and 26.7% higher than the previous BBOA respectively. It's concluded that the proposed scheme can effectively suppress the interference between users in the cognitive smart grid, and greatly improve the spectrum utilization and system energy efficiency.