Real-time safety metrics are important for automated driving systems (ADS) to assess the risk of driving situations and assist in decision-making. Although a number of real-time safety metrics have been proposed in the literature, there is a lack of systematic performance evaluations of these metrics. As different behavioral assumptions are adopted in different safety metrics, it is difficult to compare the safety metrics and evaluate their performance. To overcome this challenge, in this study, we propose an evaluation framework utilizing logged vehicle trajectory data so that vehicle trajectories for both the subject vehicle (SV) and background vehicles (BVs) are obtained and the prediction errors caused by behavioral assumptions can be eliminated. Specifically, we examine whether the SV is in a collision unavoidable situation at each moment, given all near-future trajectories of BVs. In this way, we level the ground for a fair comparison of different safety metrics, as a good safety metric should always alarm in advance to the collision unavoidable moment. When trajectory data from a large number of trips are available, we can systematically evaluate and compare different metrics’ statistical performance. In the case study, three representative real-time safety metrics, including the time-to-collision (TTC), the PEGASUS Criticality Metric (PCM) and the Model Predictive Instantaneous Safety Metric (MPrISM), are evaluated using a large-scale simulated trajectory dataset. The results demonstrate that the MPrISM achieves the highest recall and the PCM has the best accuracy. The proposed evaluation framework is important for researchers, practitioners, and regulators to characterize different metrics, and to select appropriate metrics for different applications. Moreover, by conducting failure analysis on moments when a safety metric fails, we can identify its potential weaknesses, which can be valuable for potential refinements and improvements.


    Zugriff

    Zugriff prüfen

    Verfügbarkeit in meiner Bibliothek prüfen

    Bestellung bei Subito €


    Exportieren, teilen und zitieren



    Titel :

    Evaluation of Automated Driving System Safety Metrics With Logged Vehicle Trajectory Data


    Beteiligte:
    Yan, Xintao (Autor:in) / Feng, Shuo (Autor:in) / LeBlanc, David J. (Autor:in) / Flannagan, Carol (Autor:in) / Liu, Henry X. (Autor:in)


    Erscheinungsdatum :

    01.08.2024


    Format / Umfang :

    2264785 byte




    Medientyp :

    Aufsatz (Zeitschrift)


    Format :

    Elektronische Ressource


    Sprache :

    Englisch



    GENERATING AUTONOMOUS VEHICLE SIMULATION DATA FROM LOGGED DATA

    WYRWAS JOHN / SMITH JESSICA / BOX SIMON | Europäisches Patentamt | 2021

    Freier Zugriff

    GENERATING AUTONOMOUS VEHICLE SIMULATION DATA FROM LOGGED DATA

    WYRWAS JOHN MICHAEL / SMITH JESSICA ELIZABETH / BOX SIMON | Europäisches Patentamt | 2021

    Freier Zugriff

    GENERATING AUTONOMOUS VEHICLE SIMULATION DATA FROM LOGGED DATA

    WYRWAS JOHN MICHAEL / SMITH JESSICA ELIZABETH / BOX SIMON | Europäisches Patentamt | 2025

    Freier Zugriff

    Safety Metric Aware Trajectory Repairing for Automated Driving

    Tong, Kailin / Dikic, Berin / Xiao, Wenbo et al. | IEEE | 2024


    Automated driving trajectory generating device and automated driving device

    HOTTA DAICHI / KAWANAI TAICHI / HAYASHI YUSUKE | Europäisches Patentamt | 2023

    Freier Zugriff