In order to reduce the problems of the mobile robot easily overlooking local narrow areas, high path redundancy, and low exploration efficiency during the exploration process, an autonomous exploration algorithm based on hierarchical frontier and visibility graph is proposed. Firstly, the local frontier is extracted in real-time based on the state-changed voxels in the Octomap and the global frontier is incrementally constructed, clustering the frontier to obtain candidate target points. Secondly, candidate target points are evaluated using a comprehensive index based on an incrementally updated visibility graph, by employing an evaluation function in the form of exponential decay. Thirdly, the visibility graph is integrated with the D*Lite algorithm, guided by the principles of dynamic programming, to facilitate rapid completion of exploration of unknown environments by the robot and avoid redundant paths. Finally, simulation experiments conducted in different environments show that the algorithm outperforms NBVP, GBP2 and DSVP algorithms in terms of distance traveled, runtime, and exploration efficiency. These results indicate that the algorithm effectively addresses the problems of overlooking local narrow areas and high path redundancy, thereby improving the efficiency of robot autonomous exploration.