Aircraft separation assurance is an extremely challenging task in a complex airspace. Deep Reinforcement Learning (DRL) was used to develop aircraft separation assurance models in our previous works. Though these models have shown promising performance, the DRL agents make decisions in a nontransparent way, limiting their use in safety-critical applications. In order to build a trustworthy DRL model for aircraft separation assurance, we propose a novel framework to provide stepwise explanations of agent behaviors. At a high level, our framework distills a complex DRL model into a shallow Soft Decision Tree (SDT) and uses the distilled knowledge in SDT to provide visual explanations of agent behaviors in each step. Specifically, the proposed framework incorporates (1) a distillation module to transfer knowledge from DRL policies to tree-structured policies with clear decision paths and (2) a visualization module with a graphical interface to provide visual explanations in real time. With our proposed framework, the information is extracted from the distilled SDT and illustrated with the interface. Through extensive numerical experiments in an open-source air traffic simulator with challenging environment settings, our results show that the proposed framework can support explainable decision-making for aircraft separation assurance.
Explainable Deep Reinforcement Learning for Aircraft Separation Assurance
2022-09-18
3387030 byte
Conference paper
Electronic Resource
English
A Deep Multi-Agent Reinforcement Learning Approach to Autonomous Separation Assurance
ArXiv | 2020
|