Skip to yearly menu bar Skip to main content


Poster
in
Workshop: Bayesian Deep Learning

Mixture-of-experts VAEs can disregard unimodal variation in surjective multimodal data

Jannik Wolff · Tassilo Klein · Moin Nabi · Rahul G Krishnan · Shinichi Nakajima


Abstract:

Machine learning systems are often deployed in domains that entail data from multiple modalities, for example, phenotypic and genotypic characteristics describe patients in healthcare. Previous works have developed variational autoencoders (VAEs) that generate multimodal data. We consider surjective data, where single datapoints from one modality (such as labels) describe multiple datapoints from another modality (such as images). We theoretically and empirically demonstrate that multimodal VAEs with mixture of experts posterior can struggle to capture unimodal variability in surjective data.

Chat is not available.