Variational autoencoders (VAEs) are one class of generative probabilistic latent-variable models designed for inference based on known data. They balance reconstruction and regularizer terms. A variational approximation produces an evidence lower bound (ELBO). Multiplying the regularizer term by beta provides a beta-VAE/ELBO, improving disentanglement of the latent space. However, any beta value different than unity violates the laws of conditional probability. To provide a similarly-parameterized VAE, we develop a Renyi (versus Shannon) entropy VAE, and a variational approximation RELBO that introduces a similar parameter. The Renyi VAE has an additional Renyi regularizer-like term with a conditional distribution that is not learned. The term is evaluated essentially analytically using a Singular Value Decomposition method.


    Access

    Download


    Export, share and cite



    Title :

    Resetting a fixed broken ELBO


    Contributors:

    Publication date :

    2023


    Type of media :

    Preprint


    Type of material :

    Electronic Resource


    Language :

    English




    Multimodal ELBO with Diffusion Decoders

    Wesego, Daniel / Rooshenas, Pedram | ArXiv | 2024

    Free access

    Deep Dive into Semi-Supervised ELBO for Improving Classification Performance

    Niloy, Fahim Faisal / Amin, M. Ashraful / Rahman, AKM Mahbubur et al. | ArXiv | 2021

    Free access

    The ELBO of Variational Autoencoders Converges to a Sum of Three Entropies

    Damm, Simon / Forster, Dennis / Velychko, Dmytro et al. | ArXiv | 2020

    Free access

    Self-resetting deep sea fixed-point buoy

    MO MINLING / ZHANG SHAOYONG / MO JIANHONG et al. | European Patent Office | 2023

    Free access

    RESETTING DEVICE

    European Patent Office | 2019

    Free access