In practical engineering, there are many challenges when solving multi-objective optimization problems, such as the black box characteristics and time-consuming evaluation. The traditional evolutionary optimization method is usually limited due to the expensive cost and the difficulty in obtaining solutions. To modify the deficiency, a data-driven Bayesian SVR adaptive modeling technique and a constrained multi-objective surrogate-based optimization method is proposed in the context of the small sample. The Bayesian SVR model is first utilized to replace the complex computer model, thus reducing the expensive cost of every call to the actual performance function. Then, the new design by maximizing the aggregation strategy of the constrained expected improvement matrix is chosen. Next, the sample information and the data-driven Bayesian SVR model is adaptively updated, and the optimization is fulfilled step by step. The superior characteristic of the Bayesian SVR model, that is, the powerful ability to explore the boundary and the measurement of the prediction uncertainty, ensures the prediction accuracy and provides an improvement direction for selecting the new sample. In addition, the proposed Chebyshev distance and Manhattan distance aggregation strategy has the advantages of low computational complexity and good applicability for multivariable optimization problems. Test functions and engineering examples show that: 1) The proposed method can effectively reduce expensive simulation costs and improve optimization efficiency for expensive constrained multi-objective problems in the context of the small sample; 2) The Pareto frontier of surrogate-based multi-objective optimization has a certain degree of superiority in convergence, diversity, and space dispersion.