The formation control problem is considered for a class of unknown heterogeneous nonlinear multi-agent systems. Firstly, the unknown nonlinear agent is transformed into a data model with time-varying parameters by using the full form dynamic linearization (FFDL) method. The estimation method of time-varying parameters is given. Then, a distributed model-free adaptive multi-agent formation control scheme is designed based on the FFDL data model. Finally, the effectiveness of the proposed model-free adaptive formation control scheme is verified. A multi-agent formation control experimental platform based on Python is developed by using three Nao robots. The experimental comparison results show that, using the proposed control scheme, the three robots can effectively complete the formation control task only by using local information. The control performance is better than the PID based formation control method.