Due to the limitation of focus depth of cameras, single focus imaging cannot capture clear scene for objects on different distances. Multi-focused image fusion technology can integrate images with different focus levels into a single focused image. How to get accurate focus map is a key point in a fusion process. Based on the powerful feature learning ability of convolution neural networks, a joint convolution auto-encoder network is proposed, which consists of common branches and private branches. With this hybrid structure, common branches tend to learn common features among multi-focused image pairs, and each private branch does for private features which distinguish an image from other images. Private features are used to calculate focus activity measurement, then focus map can be obtained. Guided by this map, a fusion rule is designed to fuse multi-focused images into a full focused image. Results on benchmark datasets demonstrate that compared with the state-of-art methods, the proposed method achieves more natural and clear focus effect. In terms of objective evaluation metrics, the proposed method also wins the first rank on most metrics.