Semantic segmentation plays a critical role in enabling intelligent vehicles to comprehend their surrounding environments. However, deep learning-based methods usually perform poorly in domain shift scenarios due to the lack of labeled data for training. Unsupervised domain adaptation (UDA) techniques have emerged to bridge the gap across different driving scenes and enhance model performance on unlabeled target environments. Although self-training UDA methods have achieved state-of-the-art (SOTA) results, the challenge of generating precise pseudo-labels persists. These pseudo-labels tend to favor majority classes, consequently sacrificing the performance of rare classes or small objects like traffic lights and signs. To address this challenge, we introduce SAM4UDASS, a novel approach that incorporates the Segment Anything Model (SAM) into self-training UDA methods for refining pseudo-labels. It involves Semantic-Guided Mask Labeling, which assigns semantic labels to unlabeled SAM masks using UDA pseudo-labels. Furthermore, we devise fusion strategy aimed at mitigating semantic granularity inconsistency between SAM masks and the target domain. SAM4UDASS innovatively integrate segmentation foundation model SAM with UDA for semantic segmentation in driving scenes and seamlessly complements self-training UDA methods. Extensive experiments on synthetic-to-real, normal-to-adverse, and pinhole-to-panoramic driving datasets demonstrate its effectiveness. It brings more than 3% mIoU gains on GTA5-to-Cityscapes, SYNTHIA-to-Cityscapes, and Cityscapes-to-ACDC when using DAFormer and achieves SOTA when using MIC.


    Access

    Check access

    Check availability in my library

    Order at Subito €


    Export, share and cite



    Title :

    SAM4UDASS: When SAM Meets Unsupervised Domain Adaptive Semantic Segmentation in Intelligent Vehicles


    Contributors:

    Published in:

    Publication date :

    2024-02-01


    Size :

    5909656 byte




    Type of media :

    Article (Journal)


    Type of material :

    Electronic Resource


    Language :

    English



    Unsupervised Alignment of Traffic Elements for Domain Adaptive Semantic Segmentation

    Gao, Yuan / Li, Yaochen / Zhang, Tengwen et al. | IEEE | 2024


    Threshold-Adaptive Unsupervised Focal Loss for Domain Adaptation of Semantic Segmentation

    Yan, Weihao / Qian, Yeqiang / Wang, Chunxiang et al. | IEEE | 2023


    When Visual Disparity Generation Meets Semantic Segmentation: A Mutual Encouragement Approach

    Zhang, Xiaohong / Chen, Yi / Zhang, Haofeng et al. | IEEE | 2021



    Unsupervised Domain Adaptation via Shared Content Representation for Semantic Segmentation

    Hubschneider, Christian / Birkenbach, Marius / Zollner, J. Marius | IEEE | 2021