Focusing on the problem of the basis functions selection for the linear-in-the-weights regression models, this paper proposes a model structure optimization algorithm based on singular value decomposition(SVD) and PRESS(predicted residual sums of squares) statistic. By dividing the original candidate matrix into several parts firstly, the comparison among poor candidate regressors is avoided. Based on this operation, the SVD and PRESS statistic are applied to each sub-block for candidate regressors selection. The generalization error of the model is taken as the direct target, and the candidate regressors are selected in an adaptively manner. By using SVD, the number of the candidate regressors is reduced and the regressors are orthogonal with each other, which effectively reduces the computation burden of the PRESS statistic. Simulation results show the effectiveness of the proposed method.