In this work, we provide a deterministic alternative to the stochastic variational training of generative autoencoders. We refer to these new generative autoencoders as AutoEncoders within Flows (AEF), since the encoder and decoder are defined as affine layers of an overall invertible architecture. This results in a deterministic encoding of the data, as opposed to the stochastic encoding of VAEs. The paper introduces two related families of AEFs. The first family relies on a partition of the ambient space and is trained by exact maximum-likelihood. The second family exploits a deterministic expansion of the ambient space and is trained by maximizing the log-probability in this extended space. This latter case leaves complete freedom in the choice of encoder, decoder and prior architectures, making it a drop-in replacement for the training of existing VAEs and VAE-style models. We show that these AEFs can have strikingly higher performance than architecturally identical VAEs in terms of log-likelihood and sample quality, especially for low dimensional latent spaces. Importantly, we show that AEF samples are substantially sharper than VAE samples.


    Access

    Download


    Export, share and cite



    Title :

    Deterministic training of generative autoencoders using invertible layers


    Contributors:

    Publication date :

    2022


    Remarks:

    International Conference on Learning Representations 2023


    Type of media :

    Preprint


    Type of material :

    Electronic Resource


    Language :

    English




    Perceptual Generative Autoencoders

    Zhang, Zijun / Zhang, Ruixiang / Li, Zongpeng et al. | ArXiv | 2019

    Free access

    Frugal Incremental Generative Modeling using Variational Autoencoders

    Enescu, Victor / Sahbi, Hichem | ArXiv | 2025

    Free access

    From Variational to Deterministic Autoencoders

    Ghosh, Partha / Sajjadi, Mehdi S. M. / Vergari, Antonio et al. | ArXiv | 2019

    Free access

    Deep Tracking Portfolios Using Autoencoders and Variational Autoencoders

    Urrego, Daniel Aragón / Nieto, Oscar Eduardo Reyes / Quimbayo, Carlos Andrés Zapata | Springer Verlag | 2024


    Invertible aircraft

    KEENNON MATTHEW TODD / KLINGEBIEL KARL ROBERT | European Patent Office | 2016

    Free access