Errors in the estimation of expected return of securities of portfolio selection models may have large effect on the stability of optimal strategy. A robust portfolio selection model with complicated realistic constraints is proposed. Then an improved particle swarm optimization is proposed to solve the model. The empirical analysis and comparisons from the real market data indicate that the proposed improved particle swarm optimization algorithm can solve the proposed model more efficiently, and the proposed model obtains more stable optimal portfolio strategy.