As one of the core modules of autonomous driving technology, environment perception has gradually become a hot research topic in industry and academia in recent years. However, self-driving vehicles face safety challenges due to the existence of perceptual blind spots and the lack of remote sensing capability. In this paper, a multi-modal fusion based on BEV representation for Vehicle-Infrastructure perception is proposed, referred to as V2I-BEVF, which mainly contains two branch networks for feature extraction from 2D images and 3D point clouds and transform them into BEV features, then use Deformable Attention Transformer to fuse and decode them in order to achieve high-precision real-time perception of road traffic participants. The V2I-BEVF algorithm proposed in this paper experimentally verified on the open-source roadside DAIR-V2X-I dataset from Tsinghua University and Baidu. The experimental results show that compared to several algorithm benchmarks provided by the DAIR-V2X-I dataset, the V2I-BEVF algorithm has a large improvement in pedestrian detection accuracy. Simultaneously, we verified the effectiveness of the proposed method on our collected dataset of roadside sensor devices. The V2I-BEVF algorithm can be combined with 5G/V2X communication technology and applied to V2I collaborative perception scenarios to take full advantage of wide roadside environmental perception vision and the small blind area.


    Access

    Check access

    Check availability in my library

    Order at Subito €


    Export, share and cite



    Title :

    V2I-BEVF: Multi-modal Fusion Based on BEV Representation for Vehicle-Infrastructure Perception


    Contributors:
    Xiang, Chao (author) / Xie, Xiaopo (author) / Feng, Chen (author) / Bai, Zhen (author) / Niu, Zhendong (author) / Yang, Mingchuan (author)


    Publication date :

    2023-09-24


    Size :

    7972765 byte





    Type of media :

    Conference paper


    Type of material :

    Electronic Resource


    Language :

    English



    Vehicle infrastructure collaborative perception fusion method and system

    ZHAO YIMING / MA ZE / KUMAKICHI et al. | European Patent Office | 2023

    Free access

    Multi-Modal Neural Feature Fusion for Pose Estimation and Scene Perception of Intelligent Vehicle

    Zhou, Aiguo / Pu, Jiakun / Yu, Jiangyang et al. | SAE Technical Papers | 2021


    Multi-Modal Neural Feature Fusion for Pose Estimation and Scene Perception of Intelligent Vehicle

    Zhou, Aiguo / Li, Zhenyu / Pu, Jiakun et al. | British Library Conference Proceedings | 2021


    Depth representation learning and fusion method based on multi-modal trajectory

    CHEN JING / YUAN CHANGWEI / ZHAO JIAO et al. | European Patent Office | 2023

    Free access