Although the majority of recent autonomous driving systems concentrate on developing perception methods based on ego-vehicle sensors, there is an overlooked alternative approach that involves leveraging intelligent roadside cameras to help extend the ego-vehicle perception ability beyond the visual range. We discover that most existing monocular 3D object detectors rely on the ego-vehicle prior assumption that the optical axis of the camera is parallel to the ground. However, the roadside camera is installed on a pole with a pitched angle, which makes the existing methods not optimal for roadside scenes. In this paper, we introduce a novel framework for Roadside Monocular 3D object detection with ground-aware embeddings, named MonoGAE. Specifically, the ground plane is a stable and strong prior knowledge due to the fixed installation of cameras in roadside scenarios. In order to reduce the domain gap between the ground geometry information and high-dimensional image features, we employ a supervised training paradigm with a ground plane to predict high-dimensional ground-aware embeddings. These embeddings are subsequently integrated with image features through cross-attention mechanisms. Furthermore, to improve the detector’s robustness to the divergences in cameras’ installation poses, we replace the ground plane depth map with a novel pixel-level refined ground plane equation map. Our approach demonstrates a substantial performance advantage over all previous monocular 3D object detectors on widely recognized 3D detection benchmarks for roadside cameras. The code and pre-trained models will be released soon.


    Access

    Check access

    Check availability in my library

    Order at Subito €


    Export, share and cite



    Title :

    MonoGAE: Roadside Monocular 3D Object Detection With Ground-Aware Embeddings


    Contributors:
    Yang, Lei (author) / Zhang, Xinyu (author) / Yu, Jiaxin (author) / Li, Jun (author) / Zhao, Tong (author) / Wang, Li (author) / Huang, Yi (author) / Zhang, Chuang (author) / Wang, Hong (author) / Li, Yiming (author)

    Published in:

    Publication date :

    2024-11-01


    Size :

    6763453 byte




    Type of media :

    Article (Journal)


    Type of material :

    Electronic Resource


    Language :

    English



    Monocular 3D Ray-Aware RPN For Roadside View Object Detection

    Zhang, Caiji / Tian, Bin / Sun, Yang et al. | IEEE | 2023



    Shape-Aware Monocular 3D Object Detection

    Chen, Wei / Zhao, Jie / Zhao, Wan-Lei et al. | IEEE | 2023