Lightweight models are pivotal in efficient semantic segmentation, but they often suffer from insufficient context information due to limited convolution and small receptive field. To address this problem, we propose a tailored approach to efficient semantic segmentation by leveraging two complementary distillation schemes for supplementing context information to small networks: 1) a self-attention distillation scheme, which transfers long-range context knowledge adaptively from large teacher networks to small student networks; and 2) a layer-wise context distillation scheme, which transfers structured context from deep layers to shallow layers within student networks for promoting semantic consistency of the shallow layers. Extensive experiments on the ADE20K, Cityscapes, and Camvid datasets well demonstrate the effectiveness of our proposal.


    Zugriff

    Zugriff prüfen

    Verfügbarkeit in meiner Bibliothek prüfen

    Bestellung bei Subito €


    Exportieren, teilen und zitieren



    Titel :

    Efficient Semantic Segmentation via Self-Attention and Self-Distillation


    Beteiligte:
    An, Shumin (Autor:in) / Liao, Qingmin (Autor:in) / Lu, Zongqing (Autor:in) / Xue, Jing-Hao (Autor:in)

    Erschienen in:

    Erscheinungsdatum :

    01.09.2022


    Format / Umfang :

    5131633 byte




    Medientyp :

    Aufsatz (Zeitschrift)


    Format :

    Elektronische Ressource


    Sprache :

    Englisch



    TransKD: Transformer Knowledge Distillation for Efficient Semantic Segmentation

    Liu, Ruiping / Yang, Kailun / Roitberg, Alina et al. | IEEE | 2024


    Self-Distillation Attention for Efficient and Accurate Motion Prediction in Autonomous Driving

    Lu, Ziheng / Cai, Yingfeng / Sun, Xiaoqiang et al. | IEEE | 2025


    Self-attention technology in image segmentation

    Cao, Fude / Lu, Xueyun | British Library Conference Proceedings | 2022


    Combining Semantic Self-Supervision and Self-Training for Domain Adaptation in Semantic Segmentation

    Niemeijer, Joshua / Schäfer, P. Jörg | Deutsches Zentrum für Luft- und Raumfahrt (DLR) | 2021

    Freier Zugriff