Aiming at the problem of inaccurate localization of UAVs in weak texture scenarios, we propose a UAV localization method based on edge-based acceleration. Refering to the open-source VINS-Fusion algorithmic architecture, firstly, the Harris corner point algorithm is used to extract information about the corner points that need to be optimized. The fusion of sub-pixel corner point algorithms for iterative and accuracy enhancement of extracted corner point information provides good initial values for back-end optimization threads. Secondly, we design a marginalization acceleration strategy, filtering visual residual information for optimization by sliding window method, using the Schur complement method to transform the filtered visual residual information into a priori information to be added to the optimization, splitting marginalized threads, reconstructing the information matrix, indexing rows and columns of visual residual information, moving the matrix block containing more information to the lower right corner of the information matrix, and retaining more a priori information. Finally, it is evaluated using the EuRoc dataset, and the experimental results show that, compared with the open-source vision-inertial-guidance fusion SLAM system, the proposed algorithm obtains a significant improvement in positioning accuracy, while ensuring high computational efficiency and meeting the real-time requirements of UAV positioning.