Existing infrared and visible image fusion algorithms usually extract image features from a single scale, resulting in fusion images that cannot fully retain original feature information. Aiming at the above problems, an auto-encoder network structure based on multi-scale attention mechanism is proposed to realize the fusion of infrared and visible images. Firstly, an encoder network is constructed with dense connections and multi-scale attention modules, and a self-attention mechanism is introduced to enhance the dependencies between pixels to fully extract the salient objects of infrared images and the detailed textures of visible images. The joint attention fusion network of channels and spaces further fuses the typical features of the image. In addition, a hybrid loss function based on pixels, structural similarity and color is designed to guide the network training, which further constrains the similarity between the fused image and the source image. Finally, by the subjective and objective evaluation results of the comparative experiments, it is proved that the proposed algorithm has better image fusion ability than other representative algorithms.