In recent years, deep learning-based end-to-end autonomous driving has become increasingly popular. However, deep neural networks are like black boxes. Their outputs are generally not explainable, making them not reliable to be used in real-world environments. To provide a solution to this problem, we propose an explainable deep neural network that jointly predicts driving actions and multimodal environment descriptions of traffic scenes, including bird-eye-view (BEV) maps and natural-language environment descriptions. In this network, both the context information from BEV perception and the local information from semantic perception are considered before producing the driving actions and natural-language environment descriptions. To evaluate our network, we build a new dataset with hand-labelled ground truth for driving actions and multimodal environment descriptions. Experimental results show that the combination of context information and local information enhances the prediction performance of driving action and environment description, thereby improving the safety and explainability of our end-to-end autonomous driving network.


    Zugriff

    Zugriff prüfen

    Verfügbarkeit in meiner Bibliothek prüfen

    Bestellung bei Subito €


    Exportieren, teilen und zitieren



    Titel :

    Multimodal-XAD: Explainable Autonomous Driving Based on Multimodal Environment Descriptions


    Beteiligte:
    Feng, Yuchao (Autor:in) / Feng, Zhen (Autor:in) / Hua, Wei (Autor:in) / Sun, Yuxiang (Autor:in)

    Erschienen in:

    Erscheinungsdatum :

    01.12.2024


    Format / Umfang :

    3191271 byte




    Medientyp :

    Aufsatz (Zeitschrift)


    Format :

    Elektronische Ressource


    Sprache :

    Englisch



    Multimodal End-to-End Autonomous Driving

    Xiao, Yi / Codevilla, Felipe / Gurram, Akhil et al. | IEEE | 2022


    MULTIMODAL MOTION PLANNING FRAMEWORK FOR AUTONOMOUS DRIVING VEHICLES

    ZHANG YAJIA / LI DONG / ZHANG LIANGLIANG et al. | Europäisches Patentamt | 2022

    Freier Zugriff

    Autonomous multimodal logistics

    KULKARNI GIRISH / BHASKARAN MICHAEL / GOPALAKRISHNAN VIKRANTH | Europäisches Patentamt | 2019

    Freier Zugriff

    Autonomous Multimodal Logistics

    KULKARNI GIRISH / BHASKARAN MICHAEL / GOPALAKRISHNAN VIKRANTH | Europäisches Patentamt | 2018

    Freier Zugriff

    Motion Query-based Multimodal Vehicle Trajectory Prediction for Autonomous Driving

    Jiang, Hao / Niu, Yixun / Hu, Chuan et al. | IEEE | 2023