This paper proposes a new framework for adaptive parameter estimation for discrete-time systems based on the stochastic Newton algorithm. Different to the classical parameter estimation algorithms, the convergence of the estimation error herein only requires the finite excitation(FE) condition rather than the persistent excitation(PE) condition. The main merit of the algorithm is that the original cost function is modified, where the past error information is used together with the current error information to construct the random gradient vector, so that reusing the historical information can help to relax the PE condition to the FE condition. Furthermore, to solve the problem of slow convergence of the stochastic steepest descent algorithm and avoid the potential ill-conditioned problem, the estimated inverse of the Hessian matrix is constructed as the learning gain and the stochastic Newton algorithm is used to derive the adaptive laws with its exponential convergence. Finally, the convergence of the proposed parameter estimation algorithm under different excitation conditions is evaluated via the Lyapunov theory. The effectiveness and superiority of the proposed algorithms are verified by comparative simulations.