Recently, 3D object detection techniques based on the fusion of camera and LiDAR sensor modalities have received much attention due to their complementary capabilities. How-ever, prevalent multi-modal models are relatively homogeneous in terms of feature fusion strategies, making their performance being strictly limited to the detection results of one of the modalities. While the latest data-level fusion models based on virtual point clouds do not make further use of image features, resulting in a large amount of noise in depth estimation. To address the above issues, this paper integrates the advantages of data-level and feature-level sensor fusion, and proposes MLF3D, a 3D object detection based on multi-level fusion. MLF3D generates virtual point clouds to realize the data-level fusion, and implements feature-level fusion through two key designs: VIConv3D and ASFA. VIConv3D reduces the noise problem and realizes deep interactive enhancement of features through cross-modal fusion, noise sensing, and cross-space fusion. ASFA refines the bounding box by adaptively fusing cross-layer spatial semantic information. Our MLF3D achieves 92.91%, 87.71% AP and 85.25% AP in easy, medium and hard scenarios on the KITTI’s 3D Car Detection Leaderboard, realizing excellent performance.


    Access

    Check access

    Check availability in my library

    Order at Subito €


    Export, share and cite



    Title :

    MLF3D: Multi-Level Fusion for Multi-Modal 3D Object Detection


    Contributors:
    Jiang, Han (author) / Wang, Jianbin (author) / Xiao, Jianru (author) / Zhao, Yanan (author) / Chen, Wanqing (author) / Ren, Yilong (author) / Yu, Haiyang (author)


    Publication date :

    2024-06-02


    Size :

    2769536 byte





    Type of media :

    Conference paper


    Type of material :

    Electronic Resource


    Language :

    English



    MMF-Track: Multi-Modal Multi-Level Fusion for 3D Single Object Tracking

    Li, Zhiheng / Cui, Yubo / Lin, Yu et al. | IEEE | 2024


    Frustum FusionNet: Amodal 3D Object Detection with Multi-Modal Feature Fusion

    Zuo, Liangyu / Li, Yaochen / Han, Mengtao et al. | IEEE | 2021


    PPF-Det: Point-Pixel Fusion for Multi-Modal 3D Object Detection

    Xie, Guotao / Chen, Zhiyuan / Gao, Ming et al. | IEEE | 2024


    Multi-Modal Sensor Fusion and Object Tracking for Autonomous Racing

    Karle, Phillip / Fent, Felix / Huch, Sebastian et al. | IEEE | 2023


    Improving multi-modal data fusion by anomaly detection

    Simanek, J. | British Library Online Contents | 2015