As an important social media platform, analyzing, detecting and tracking the important social events in microblog can provide public issues in time. However, due to the fragmentation, heterogeneity and real-time characteristics of microblog, traditional techniques can hardly analyze mass microblog efficiently. Therefore, a social event detection and tracking framework based on multimodal feature deep fusion is proposed. Firstly, in the framework, events in microblogs are labeled by text process. Then, the detection and description of events are achieved by multimodal feature deep fusion. Finally, the tracking of the event stream is accomplished by the graph variation based on time smooth. The experiments in a real dataset show that the proposed method can detect and track events in the microblog stream effectively.