Comprehensive and accurate perception of the real 3D world is the basis of autonomous driving. However, many perceptual methods focus on a single task or object type, and the accuracy of existing multi-task or multi-object methods is difficult to balance against their real-time performance. This paper presents a unified framework for concurrent dynamic multi-object joint perception, which introduces a real-time monocular joint perception network termed MJPNet. In MJPNet relative weightings are automatically learned by a series of developed network branches. By training an end-to-end deep convolutional neural network on a shared feature encoder and many proposed decoding sub-branches, the information of the 2D category and 3D position/pose/size of an object are reconstructed both simultaneously and accurately. Moreover, the effective information among subtasks is transferred by multi-stream learning, guaranteeing the accuracy of each task. Compared to various state-of-the-arts, comprehensive evaluations on the benchmark of challenging image sequences demonstrate the superior performance of our 2D detection and 3D reconstruction of depth, lateral distance, orientation, and heading angle. Moreover, on the KITTI test set, the real-time runtime (up to 15 fps) of MJPNet significantly outran the public state-of-the-art visual detection methods. Accompanying video: https://youtu.be/Z-goToOlI94.


    Access

    Check access

    Check availability in my library

    Order at Subito €


    Export, share and cite



    Title :

    Real-Time Monocular Joint Perception Network for Autonomous Driving


    Contributors:
    Li, Keqiang (author) / Xiong, Hui (author) / Liu, Jinxin (author) / Xu, Qing (author) / Wang, Jianqiang (author)


    Publication date :

    2022-09-01


    Size :

    4966174 byte




    Type of media :

    Article (Journal)


    Type of material :

    Electronic Resource


    Language :

    English





    MultiNet: Real-time Joint Semantic Reasoning for Autonomous Driving

    Teichmann, Marvin / Weber, Michael / Zollner, Marius et al. | IEEE | 2018


    Federated Monocular 3D Object Detection for Autonomous Driving

    Chi, Fangyuan / Wang, Yixiao / Nasiopoulos, Panos et al. | TIBKAT | 2023

    Free access

    MULTINET: REAL-TIME JOINT SEMANTIC REASONING FOR AUTONOMOUS DRIVING

    Teichmann, Marvin / Weber, Michael / Zöllner, Marius et al. | British Library Conference Proceedings | 2018