Mixtures of Variational Autoencoders

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

In this paper, we develop a new deep mixture learning framework, aiming to learn underlying complex data structures. Each component in the mixture model is implemented using a Variational Autoencoder (VAE). VAE is a well known deep
learning model which models a latent space data representation on a variational manifold. The mixing parameters are estimated from a Dirichlet distribution modelled by each encoder. In order to train this mixture model, named M-VAE, we derive a mixture evidence lower bound on the sample log-likelihood, which is optimized in order to jointly estimate all mixture components. We further propose to use the d-variables Hilbert-Schmidt Independence Criterion (dHSIC) as a regularization criterion in order to enforce the independence among the encoders’ distributions. This criterion encourages the proposed mixture components to learn different data distributions and represent them in the latent space. During the experiments with the proposed M-VAE model we observe that it can be used for discovering disentangled data representations which can not be achieved with a single VAE.
Original languageEnglish
Title of host publicationProc. Int. Conf. on Image Processing, Theory, Tools and Applications (IPTA)
Place of PublicationParis, France
PublisherIEEE
Number of pages6
DOIs
Publication statusPublished - 10 Nov 2020

Cite this