In response to the growing demand for more intelligent and efficient methods of analyzing aviation accidents, this paper explores an AI-based approach to data mining and utilization in aviation accident investigation reports. Based on the framework of Retrieval-Augmented Generation (RAG), we constructed a benchmark dataset of aviation accident reports, we also built an external knowledge base from the Air Transport Association Specification 100 (ATA100) and enhanced the content with components in Aircraft Manual Maintenance (AMM). Experiments of sensitive factors were conducted to find out the most influential factors of RAG. The results show that the LLM, the retrieved number of external knowledge segments (top-k), and especially the embedding model have the greatest influence on RAG. The optimal factors reached scores of 0.75 in recall coverage rate, 0.74 in two-digit codes accuracy and 0.53 in four-digit codes accuracy. Based on the optimal sensitive factors, we conducted experiment of external knowledge enhancement. The results indicate that a more comprehensive external knowledge base can benefit both LLM and embedding model as recall coverage rate improved 8%, two-digit codes accuracy improved 11%, and four-digit codes improved 16%. The outcome of our work indicates that the RAG-base analysis method can significantly enhance the efficiency and accuracy of aviation knowledge extraction task while being highly automated and self-iterative, demonstrating its potential for future applications in the aviation industry.


    Access

    Check access

    Check availability in my library

    Order at Subito €


    Export, share and cite



    Title :

    A Retrieval-Augmented Generation-Based Method for Aviation Accident Data Analysis


    Contributors:


    Publication date :

    2024-12-27


    Size :

    1649719 byte




    Type of media :

    Conference paper


    Type of material :

    Electronic Resource


    Language :

    English



    Aviation Accident Causal Analysis

    Ford, Carl N. / Sandusky, Robert / Jack, Todd D. et al. | SAE Technical Papers | 1999