There have been many recent advancements in imitation and reinforcement learning for autonomous driving, but existing metrics generally lack the means to capture a wide range of driving behaviors and compare the severity of different failure cases. To address this shortcoming, we introduce Quan-titative Evaluation for Driving (QED), which assesses different aspects of driving behavior including the ability to stay in the center of the lane, avoid weaving and erratic behavior, follow the speed limit, and avoid collisions. We compare scores generated by QED against scores assigned by human evaluators on 30 different drivers and 6 different towns in the CARLA driving simulator. In "easy" evaluation scenarios where better drivers are easily distinguished from worse drivers, QED attains 0.96 Pearson correlation and 0.97 Spearman correlation with human evaluators, similar to the baseline inter-human-evaluator 0.96 Pearson correlation and 0.95 Spearman correlation. In "hard" evaluation scenarios where ranking drivers is more ambiguous, QED attains 0.84 Pearson correlation and 0.74 Spearman correlation with human evaluators, slighter higher than the baseline inter-human-evaluator 0.78 Pearson correlation and 0.7 Spearman correlation. While QED may not capture every characteristic that defines good driving, we consider it an important foundation for reproducibility and standardization in the community.


    Zugriff

    Zugriff prüfen

    Verfügbarkeit in meiner Bibliothek prüfen

    Bestellung bei Subito €


    Exportieren, teilen und zitieren



    Titel :

    Quantitative Evaluation of Autonomous Driving in CARLA


    Beteiligte:
    Gao, Shang (Autor:in) / Paulissen, Spencer (Autor:in) / Coletti, Mark (Autor:in) / Patton, Robert (Autor:in)


    Erscheinungsdatum :

    11.07.2021


    Format / Umfang :

    987101 byte




    Medientyp :

    Aufsatz (Konferenz)


    Format :

    Elektronische Ressource


    Sprache :

    Englisch



    Injecting Conflict Situations in Autonomous Driving Simulation Using CARLA

    Mihaylova, Tsvetomila / Reitmann, Stefan / Topp, Elin A. et al. | IEEE | 2025


    Injecting Conflict Situations in Autonomous Driving Simulation using CARLA

    Mihaylova, Tsvetomila / Reitmann, Stefan / Topp, Elin A. et al. | ArXiv | 2025

    Freier Zugriff

    ANTI-CARLA: An Adversarial Testing Framework for Autonomous Vehicles in CARLA

    Ramakrishna, Shreyas / Luo, Baiting / Kuhn, Christopher B. et al. | IEEE | 2022


    CARLA Real Traffic Scenarios -- novel training ground and benchmark for autonomous driving

    Osiński, Błażej / Miłoś, Piotr / Jakubowski, Adam et al. | ArXiv | 2020

    Freier Zugriff

    Rationale-aware Autonomous Driving Policy utilizing Safety Force Field implemented on CARLA Simulator

    Suk, Ho / Kim, Taewoo / Park, Hyungbin et al. | ArXiv | 2022

    Freier Zugriff