Abstract Applications based on artificial intelligence (AI) promise benefits, ranging from improved performance to increased capabilities in many industries. In the aviation domain, one example is the new Airborne Collision Avoidance System (ACAS X). The current investigation aims at combining ACAS X and AI to maintain its performance while decreasing the memory footprint. However, the anticipation of AI being increasingly used confronts regulators with challenges in terms of safety assurance and certification. Consequently, the European Union Aviation Safety Agency (EASA) published a concept paper for machine learning applications in aviation. Both, the Concept of Operation (ConOps) in combination with an Operational Design Domain (ODD), are listed as objectives to be met for the safety analysis. From a developer’s perspective, this raises questions on how to effectively derive the ODD from ConOps and test the given system based on the ODD description. Based on an exemplary use case of a Near Mid-Air Collision avoidance between two aircraft through the advisories of ACAS X, a highly automated framework for generating and testing synthetic data is proposed. Using this framework, 1800 Near Mid-Air Collision scenario files are created and automatically executed in the simulation environment FlightGear. Scenario-based testing is used for the logging of ACAS X advisory data and evaluating it against predefined requirements. By this approach, an efficient way of verifying system requirements and conducting automated testing based on the ODD definition is demonstrated. Throughout this process, Model-Based Systems Engineering (MBSE) is used to reduce and manage complexity. The framework in this paper enables a systematic and highly automated approach for scenario generation based on the ODD.


    Access

    Download


    Export, share and cite



    Title :

    Automated scenario generation from Operational Design Domain model for testing AI-based systems in aviation


    Additional title:

    CEAS Aeronaut J



    Published in:

    CEAS Aeronautical Journal ; 16 , 1 ; 197-212


    Publication date :

    2025-01-01




    Type of media :

    Article (Journal)


    Type of material :

    Electronic Resource


    Language :

    English




    Scenario Coverage of Operational Design Domain for AI-Based Systems in Aviation

    Gupta, Siddhartha / Sprockhoff, Jasper / Durak, Umut | AIAA | 2025


    Operational Domain Metamodel for Testing AI Systems in Aviation

    Gupta, Siddhartha / Durak, Umut | AIAA | 2023


    From Operational Design Domain to Runtime Monitoring of AI-Based Aviation Systems

    Torens, Christoph / Gupta, Siddhartha / Roy, Nirmal et al. | IEEE | 2024


    Scenario-Based Testing of Real-Time Embedded Systems in Aviation

    Munkoh-Buabeng, Edwin / Gupta, Siddhartha / Durak, Umut et al. | AIAA | 2025