This article presents a technique for trajectory planning based on parameterized high-level actions. These high-level actions are subtrajectories that have variable shape and duration. The use of high-level actions can improve the performance of guidance algorithms. Specifically, we show how the use of high-level actions improves the performance of guidance policies that are generated via reinforcement learning (RL). RL has shown great promise for solving complex control, guidance, and coordination problems but can still suffer from long training times and poor performance. This work shows how the use of high-level actions reduces the required number of training steps and increases the path performance of an RL-trained guidance policy. We demonstrate the method on a space-shuttle guidance example. We show the proposed method increases the path performance (latitude range) by 18% compared with a baseline RL implementation. Similarly, we show the proposed method achieves steady state during training with approximately 75% fewer training steps. We also show how the guidance policy enables effective performance in an obstacle field. Finally, this article develops a loss function term for policy-gradient-based deep RL, which is analogous to an antiwindup mechanism in feedback control. We demonstrate that the inclusion of this term in the underlying optimization increases the average policy return in our numerical example.


    Zugriff

    Zugriff prüfen

    Verfügbarkeit in meiner Bibliothek prüfen

    Bestellung bei Subito €


    Exportieren, teilen und zitieren



    Titel :

    Trajectory Planning With Deep Reinforcement Learning in High-Level Action Spaces


    Beteiligte:
    Williams, Kyle R. (Autor:in) / Schlossman, Rachel (Autor:in) / Whitten, Daniel (Autor:in) / Ingram, Joe (Autor:in) / Musuvathy, Srideep (Autor:in) / Pagan, James (Autor:in) / Williams, Kyle A. (Autor:in) / Green, Sam (Autor:in) / Patel, Anirudh (Autor:in) / Mazumdar, Anirban (Autor:in)


    Erscheinungsdatum :

    01.06.2023


    Format / Umfang :

    2470233 byte




    Medientyp :

    Aufsatz (Zeitschrift)


    Format :

    Elektronische Ressource


    Sprache :

    Englisch



    Crossroad lighthouse trajectory planning method based on deep reinforcement learning

    LI BOQI / ZHANG WEIWEI / CHEN JIANG et al. | Europäisches Patentamt | 2024

    Freier Zugriff

    3-D Autonomous Entry Trajectory Planning via Hybrid Action Reinforcement Learning

    Peng, Gaoxiang / Wang, Bo / Liu, Lei et al. | IEEE | 2025


    Trajectory Planning for Automated Parking Systems Using Deep Reinforcement Learning

    Du, Zhuo / Miao, Qiheng / Zong, Changfu | Springer Verlag | 2020


    Deep Reinforcement Learning-Based 3D Trajectory Planning for Cellular Connected UAV

    Xiang Liu / Weizhi Zhong / Xin Wang et al. | DOAJ | 2024

    Freier Zugriff

    Trajectory Planning of UAV in Unknown Dynamic Environment with Deep Reinforcement Learning

    Wang, Jia / Wang, Weihong / Wu, Qian | Springer Verlag | 2019