Normative modelling is an emerging method for understanding the underlying heterogeneity within brain disorders like Alzheimer Disease (AD) by quantifying how each patient deviates from the expected normative pattern that has been learned from a healthy control distribution. Existing deep learning based normative models on magnetic resonance imaging (MRI) neuroimaging data use unimodal autoencoders with a single encoder and decoder that may fail to capture the relationship between brain measurements extracted from different MRI modalities. In this work, we propose multi-modal variational autoencoder (mmVAE) based normative modelling framework that can capture the joint distribution between different modalities and apply it for normative modeling. The deviation maps generated by our proposed multimodal model (mmVAE) are more sensitive to disease staging within AD, have a better correlation with patient cognition and higher number of brain regions with statistically significant deviations compared to a unimodal baseline model with all modalities concatenated as a single input.