Freespace detection is an important part of autonomous driving technology. Compared with structured on-road scenes, unstructured off-road scenes face more challenges. Multi-modal fusion method is a viable solution to these challenges. But existing fusion methods do not fully utilize the multi-modal features. In this paper, we propose an effective multi-modal network named M2F2-Net for freespace detection in unstructured off-road scenes. We propose a multi-modal feature fusion strategy named Multi-modal Cross Fusion (MCF). MCF module is simple but effective in fusing the features of RGB images and surface normal maps. Meanwhile, a multi-modal segmentation decoder module is designed to decouple the segmentation of two modalities, and it further helps the features of both modalities to be fully utilized. In order to solve the problem that the road edge is difficult to extract in the unstructured scenes, we also propose an edge segmentation decoder module. Extensive experiments show that our approach can lead to significant improvements, which brings 6.1% F1 and 10.8% IoU improvements. Our code will be available at https://github.com/yhl1010/M2F2-Net.


    Access

    Check access

    Check availability in my library

    Order at Subito €


    Export, share and cite



    Title :

    M2F2-Net: Multi-Modal Feature Fusion for Unstructured Off-Road Freespace Detection


    Contributors:
    Ye, Hongliang (author) / Mei, Jilin (author) / Hu, Yu (author)


    Publication date :

    2023-06-04


    Size :

    3160808 byte





    Type of media :

    Conference paper


    Type of material :

    Electronic Resource


    Language :

    English



    B-spline modeling of road surfaces for freespace estimation

    Wedel, Andreas / Franke, Uwe / Badino, Hernan et al. | IEEE | 2008


    AUTONOMOUS VEHICLE WITH FREESPACE PLANNER

    ZHOU XIAOYU / KUSHWAHA SHREYANS / FASOLA JUAN | European Patent Office | 2025

    Free access


    Frustum FusionNet: Amodal 3D Object Detection with Multi-Modal Feature Fusion

    Zuo, Liangyu / Li, Yaochen / Han, Mengtao et al. | IEEE | 2021


    Unstructured road detection via combining the model-based and feature-based methods

    Qi, Naixin / Yang, Xiaogang / Li, Chuanxiang et al. | IET | 2019

    Free access