Multimodality fusion based on deep neural networks (DNN) is a significant method for intelligent vehicles. The special characteristics of DNN lead to the issue of AI safety and safety test. In this paper, we firstly propose a multimodality fusion framework called Integrated Multimodality Fusion Deep Neural Network (IMF-DNN), which can flexibly accomplish both object detection and end-to-end driving policy for prediction of steering angle and speed. Then, we propose a DNN safety test strategy, which systematically analyzes DNN's robustness and generalization ability in large amounts of diverse driving environment conditions. The test in this paper is based on our IMF-DNN model and the strategy can be widely used for other DNNs. Finally, the experiment analysis is performed on KITTI for object detection and the dateset DBNet for end-to-end tasks. The results show the superior accuracy of the proposed IMF-DNN model and the test strategy's potential ability to improve the robustness and generalization of autonomous vehicle deep learning model. Code is available at https://github.com/ennisnie/IMF-DNN


    Zugriff

    Zugriff prüfen

    Verfügbarkeit in meiner Bibliothek prüfen

    Bestellung bei Subito €


    Exportieren, teilen und zitieren



    Titel :

    A Multimodality Fusion Deep Neural Network and Safety Test Strategy for Intelligent Vehicles


    Beteiligte:
    Nie, Jian (Autor:in) / Yan, Jun (Autor:in) / Yin, Huilin (Autor:in) / Ren, Lei (Autor:in) / Meng, Qian (Autor:in)

    Erschienen in:

    Erscheinungsdatum :

    01.06.2021


    Format / Umfang :

    4937884 byte




    Medientyp :

    Aufsatz (Zeitschrift)


    Format :

    Elektronische Ressource


    Sprache :

    Englisch



    Experiments in multimodality image classification and data fusion

    Farag, A.A. / Mohamed, R. / Mahdi, H. | IEEE | 2002


    Experiments in Multimodality Image Classification and Data Fusion

    Farag, A. / Mahdi, H. / Mohamed, R. et al. | British Library Conference Proceedings | 2002


    Multimodality

    Nobis, Claudia | Transportation Research Record | 2007


    Supervised Hash Coding With Deep Neural Network for Environment Perception of Intelligent Vehicles

    Yan, Chenggang / Xie, Hongtao / Yang, Dongbao et al. | IEEE | 2018