Real-world artificial intelligence (AI) applications often require multiple agents to work together, and effective communication and coordination between artificial agents is an indispensable step toward universal artificial intelligence. This paper takes the self-developed virtual environment for police training as a test scenario. Setting tasks requires multiple different service agent teams to cooperate or fight against each other. In order to ensure that the communication method is effective and scalable, this paper proposes the mixed deep deterministic policy gradient (Mi-DDPG) algorithm. Firstly, the bidirectional recurrent neural networks (BRNN) is added to the Actor network as the information exchange layer of the same type of agent, and then the other agent information is added to the Critic network to learn the multi-agent cooperation strategy. In addition, in order to alleviate the training pressure, the centralized training and distributed execution framework are adopted, and the Q function in the Critic network is modularized. In the experiment, the Mi-DDPG algorithm is compared with other algorithms in different scenarios, which shows its most advanced performance and potential value in real-world.