We present a conditional variational auto-encoder (VAE) which, to avoid the substantial cost of training from scratch, uses an architecture and training objective capable of leveraging a foundation model in the form of a pretrained unconditional VAE. To train the conditional VAE, we only need to train an artifact to perform amortized inference over the unconditional VAE's latent variables given a conditioning input. We demonstrate our approach on tasks including image inpainting, for which it outperforms state-of-the-art GAN-based approaches at faithfully representing the inherent uncertainty. We conclude by describing a possible application of our inpainting model, in which it is used to perform Bayesian experimental design for the purpose of guiding a sensor.


    Access

    Download


    Export, share and cite



    Title :

    Conditional Image Generation by Conditioning Variational Auto-Encoders


    Contributors:

    Publication date :

    2021


    Remarks:

    37 pages, 20 figures


    Type of media :

    Preprint


    Type of material :

    Electronic Resource


    Language :

    English




    Adversarial Training of Variational Auto-encoders for High Fidelity Image Generation

    Khan, Salman H. / Hayat, Munawar / Barnes, Nick | ArXiv | 2018

    Free access

    Multi-Stage Variational Auto-Encoders for Coarse-to-Fine Image Generation

    Cai, Lei / Gao, Hongyang / Ji, Shuiwang | ArXiv | 2017

    Free access

    Consistency Regularization for Variational Auto-Encoders

    Sinha, Samarth / Dieng, Adji B. | ArXiv | 2021

    Free access

    Variational auto-encoders with Student's t-prior

    Abiri, Najmeh / Ohlsson, Mattias | ArXiv | 2020

    Free access

    Dora: Sampling and Benchmarking for 3D Shape Variational Auto-Encoders

    Chen, Rui / Zhang, Jianfeng / Liang, Yixun et al. | ArXiv | 2024

    Free access