Mechanistic interpretability of deep learning models has emerged as a crucial research direction for understanding the functioning of neural networks. While significant progress has been made in interpreting discriminative models like transformers, understanding generative models such as Variational Autoencoders (VAEs) remains challenging. This paper introduces a comprehensive causal intervention framework for mechanistic interpretability of VAEs. We develop techniques to identify and analyze "circuit motifs" in VAEs, examining how semantic factors are encoded, processed, and disentangled through the network layers. Our approach uses targeted interventions at different levels: input manipulations, latent space perturbations, activation patching, and causal mediation analysis. We apply our framework to both synthetic datasets with known causal relationships and standard disentanglement benchmarks. Results show that our interventions can successfully isolate functional circuits, map computational graphs to causal graphs of semantic factors, and distinguish between polysemantic and monosemantic units. Furthermore, we introduce metrics for causal effect strength, intervention specificity, and circuit modularity that quantify the interpretability of VAE components. Experimental results demonstrate clear differences between VAE variants, with FactorVAE achieving higher disentanglement scores (0.084) and effect strengths (mean 4.59) compared to standard VAE (0.064, 3.99) and Beta-VAE (0.051, 3.43). Our framework advances the mechanistic understanding of generative models and provides tools for more transparent and controllable VAE architectures.


    Zugriff

    Download


    Exportieren, teilen und zitieren



    Titel :

    Causal Intervention Framework for Variational Auto Encoder Mechanistic Interpretability


    Beteiligte:
    Roy, Dip (Autor:in)

    Erscheinungsdatum :

    2025


    Medientyp :

    Preprint


    Format :

    Elektronische Ressource


    Sprache :

    Englisch




    AVAE: Adversarial Variational Auto Encoder

    Plumerault, Antoine / Borgne, Hervé Le / Hudelot, Céline | ArXiv | 2020

    Freier Zugriff

    Variational auto encoder for mixed data types

    Europäisches Patentamt | 2020

    Freier Zugriff

    Disentangled Variational Auto-Encoder for Semi-supervised Learning

    Li, Yang / Pan, Quan / Wang, Suhang et al. | ArXiv | 2017

    Freier Zugriff

    Disentangling Latent Factors of Variational Auto-Encoder with Whitening

    Hahn, Sangchul / Choi, Heeyoul | ArXiv | 2018

    Freier Zugriff

    Item-based Variational Auto-encoder for Fair Music Recommendation

    Park, Jinhyeok / Kim, Dain / Kim, Dongwoo | ArXiv | 2022

    Freier Zugriff