Abstract: In the study, Deep Boltzmann Machine (DBM) based on multimodal learning algorithm was used to analyze microblog image and textual data. The model can transform low-level features of images and texts to sparse high-level abstract concepts. A joint representation layer was employed to fuse common features derived from the two different input modalities. Additionally, the model can detect that the input characteristic of two different model data at low-level was non-linear relations. The experiment results suggested that the proposed method is effective.