Multimodal sentiment analysis and emotion recognition have become an increasingly popular research topic, with a large number of emerging of various images and videos on internet. Compared with the unimode sentiment analysis, one of the key issues in multimodal sentiment analysis is the fusion strategy. Inspired by the emotional arousal model in cognitive science, a deep emotional arousal network(DEAN) is proposed, which can simulate emotional coherence. The proposed DEAN consists of three parts: A crossmodal transformer module that simulates the functions of a human perception analysis system, a multimodal BiLSTM system that simulates the cognitive comparator, and a multimodal gating module that is used to simulate the activation mechanism in the physiological emotional arousal model. Extensive experimental comparisons on three benchmarks for multimodal sentiment analysis and emotion recognition are conducted, and the experimental results show that the DEAN outperforms the state-of-the-art methods.