Interconnection between multiple microgrids plays an important role on enhancing stability of microgrids and renewable energy utilization. Considering the needs of collaborative operation of intelligent terminals and power mutual assistance between microgrids, a real-time collaborative terminal-microgrid optimization algorithm based on multi-agent reinforcement learning is proposed. The proposed algorithm can flexibly adapt to the system uncertainties and topology changes, and optimizes energy interaction between microgrids and collaboration among intelligent terminals. Firstly, the structure of the collaborative terminal-microgird optimization system is established, and the corresponding optimization model is formulated. Then, a multi-agent Markov decision process based reinforcement learning model is proposed. Furthermore, a distributed collaborative optimization algorithm based on the multi-agent proximal policy optimization(MAPPO) algorithm is proposed. Considering the power balance constraint, a novel power balance feedback is designed to effectively avoid the occurrence of power imbalance. Simulations are conducted under three typical scenarios, and the results show that approximate global optimal solutions can be obtained without global state observation. What is more, the proposed algorithm is compared with the distributed reinforcement learning algorithms with fully state observation and partial state observation, respectively. Results show that the proposed method can achieve good collaborative optimization results and meet the requirements on compute efficiency of real-time optimization.