Variational Autoencoders (VAEs), as a form of deep generative model, have been widely used in recent years, and shown great great peformance in a number of different domains, including image generation and anomaly detection, etc.. This paper aims to explore neural network model compression method based on VAE. The experiment uses different neural network models for MNIST recognition as compression targets, including Feedforward Neural Network (FNN), Convolutional Neural Network (CNN), Recurrent Neural Network (RNN) and Long Short-Term Memory (LSTM). These models are the most basic models in deep learning, and other more complex and advanced models are based on them or inherit their features and evolve. In the experiment, the first step is to train the models mentioned above, each trained model will have different accuracy and number of total parameters. And then the variants of parameters for each model are processed as training data in VAEs separately, and the trained VAEs are tested by the true model parameters. The experimental results show that using the latent space as a representation of the model compression can improve the compression rate compared to some traditional methods such as pruning and quantization, meanwhile the accuracy is not greatly affected using the model parameters reconstructed based on the latent space. In the future, a variety of different large-scale deep learning models will be used more widely, so exploring different ways to save time and space on saving or transferring models will become necessary, and the use of VAE in this paper can provide a basis for these further explorations.


    Access

    Download


    Export, share and cite



    Title :

    Variational autoencoder-based neural network model compression


    Contributors:
    Cheng, Liang (author) / Guan, Peiyuan (author) / Taherkordi, Amir (author) / Liu, Lei (author) / Lan, Dapeng (author)

    Publication date :

    2024


    Type of media :

    Preprint


    Type of material :

    Electronic Resource


    Language :

    English




    Variational Autoencoder

    Pinheiro Cinelli, Lucas / Araújo Marins, Matheus / Barros da Silva, Eduardo Antúnio et al. | Springer Verlag | 2021


    Variational Autoencoder

    Okadome, Takeshi | Springer Verlag | 2025


    Autoencoding Variational Autoencoder

    Cemgil, A. Taylan / Ghaisas, Sumedh / Dvijotham, Krishnamurthy et al. | ArXiv | 2020

    Free access

    Variational Lossy Autoencoder

    Chen, Xi / Kingma, Diederik P. / Salimans, Tim et al. | ArXiv | 2016

    Free access

    Branched Variational Autoencoder Classifiers

    Salah, Ahmed / Yevick, David | ArXiv | 2024

    Free access