3D object detection is a core component of automated driving systems. State-of-the-art methods fuse RGB imagery and LiDAR point cloud data frame-by-frame for 3D bounding box regression. However, frame-by-frame 3D object detection suffers from noise, field-of-view obstruction, and sparsity. We propose a novel Temporal Fusion Module (TFM) to use information from previous time-steps to mitigate these problems. First, a state-of-the-art frustum network extracts point cloud features from raw RGB and LiDAR point cloud data frame-by-frame. Then, our TFM module fuses these features with a recurrent neural network. As a result, 3D object detection becomes robust against single frame failures and transient occlusions. Experiments on the KITTI object tracking dataset show the efficiency of the proposed TFM, where we obtain 6%, 4%, and 6% improvements on Car, Pedestrian, and Cyclist classes, respectively, compared to frame-by-frame baselines. Furthermore, ablation studies reinforce that the subject of improvement is temporal fusion and show the effects of different placements of TFM in the object detection pipeline. Our code is open-source and available at https://github.com/emecercelik/Temp-Frustum-Net.git.


    Access

    Check access

    Check availability in my library

    Order at Subito €


    Export, share and cite



    Title :

    Temp-Frustum Net: 3D Object Detection with Temporal Fusion


    Contributors:


    Publication date :

    2021-07-11


    Size :

    3602639 byte




    Type of media :

    Conference paper


    Type of material :

    Electronic Resource


    Language :

    English



    TEMP-FRUSTUM NET: 3D OBJECT DETECTION WITH TEMPORAL FUSION

    Erçelik, Emeç / Yurtsever, Ekim / Knoll, Alois | British Library Conference Proceedings | 2021


    Frustum FusionNet: Amodal 3D Object Detection with Multi-Modal Feature Fusion

    Zuo, Liangyu / Li, Yaochen / Han, Mengtao et al. | IEEE | 2021


    Faraway-Frustum: Dealing with Lidar Sparsity for 3D Object Detection using Fusion

    Zhang, Haolin / Yang, Dongfang / Yurtsever, Ekim et al. | IEEE | 2021


    Vehicle lamp system with frustum reflector and lighting method using frustum reflector

    CHEN LINSHENG / KUMAR ARUN / TEODECKI JOHN et al. | European Patent Office | 2024

    Free access

    High Dimensional Frustum PointNet for 3D Object Detection from Camera, LiDAR, and Radar

    Wang, Leichen / Chen, Tianbai / Anklam, Carsten et al. | IEEE | 2020