Semantic segmentation of RGB-T images is a complex task due to the challenges involved in fusing information from multi-modalities, which requires significant computational resources. This paper presents a novel and lightweight network architecture for RGB-T semantic segmentation that incorporates a parameter-free feature fusion module to integrate complementary information from different modalities. Furthermore, we propose a pretrained parameter selection strategy to improve convergence speed and accuracy. The network is designed to be computationally efficient and lightweight, making it well-suited for real-time applications. Our network architecture employs middle fusion techniques to extract features with separate encoders from the different modalities, and then a parameter-free cross-modal attention mechanism is designed to selectively connect the most relevant information from each modality. Additionally, we investigate the impact of pretrained parameter selection on the performance of the network. Experimental results on an urban scene dataset demonstrate that our approach outperforms real-time state-of-the-art methods in the literature while showing comparable performance with state-of-the-art methods that require up to 100 times the computational complexity. Our findings highlight the potential of the RGB-T fusion-based semantic segmentation for applications in real-world scenarios.


    Access

    Check access

    Check availability in my library

    Order at Subito €


    Export, share and cite



    Title :

    A Lightweight RGB-T Fusion Network for Practical Semantic Segmentation


    Contributors:
    Zhang, Haoyuan (author) / Li, Zifeng (author) / Wu, Zhenyu (author) / Wang, Danwei (author)


    Publication date :

    2023-09-24


    Size :

    8350812 byte





    Type of media :

    Conference paper


    Type of material :

    Electronic Resource


    Language :

    English




    Lightweight Real-Time Semantic Segmentation Network With Efficient Transformer and CNN

    Xu, Guoan / Li, Juncheng / Gao, Guangwei et al. | IEEE | 2023




    Lidar Mapping Optimization Based on Lightweight Semantic Segmentation

    Zhao, Zhihao / Zhang, Wenquan / Gu, Jianfeng et al. | IEEE | 2019