In autonomous driving, the ego vehicle and its surrounding traffic environments always have uncertainties like parameter and structural errors, behavior randomness of road users, etc. Furthermore, environmental sensors are noisy or even biased. This problem can be formulated as a partially observable Markov decision process. Existing methods lack a good representation of historical information, making it very challenging to find an optimal policy. This paper proposes a belief state separated reinforcement learning (RL) algorithm for decision-making of autonomous driving in uncertain environments. We extend the separation principle from linear Gaussian systems to general nonlinear stochastic environments, where the belief state, defined as the posterior distribution of the true state, is found to be a sufficient statistic of historical information. This belief state is estimated by action-enhanced variational inference from historical information and is proved to satisfy the Markovian property, thus allowing us to obtain the optimal policy using traditional RL algorithms for Markov decision processes. The policy gradient of a task-specific prior model is mixed with that of the interaction data to improve learning performance. The proposed algorithm is evaluated in a multi-lane autonomous driving task, where the surrounding vehicles are subject to behavior uncertainty and observation noise. The simulation results show that compared with existing RL algorithms, the proposed method can achieve a higher average return with better driving performance.


    Zugriff

    Zugriff prüfen

    Verfügbarkeit in meiner Bibliothek prüfen

    Bestellung bei Subito €


    Exportieren, teilen und zitieren



    Titel :

    Belief state separated reinforcement learning for autonomous vehicle decision making under uncertainty


    Beteiligte:
    Gu, Ziqing (Autor:in) / Yang, Yujie (Autor:in) / Duan, Jingliang (Autor:in) / Li, Shengbo Eben (Autor:in) / Chen, Jianyu (Autor:in) / Cao, Wenhan (Autor:in) / Zheng, Sifa (Autor:in)


    Erscheinungsdatum :

    19.09.2021


    Format / Umfang :

    1005283 byte




    Medientyp :

    Aufsatz (Konferenz)


    Format :

    Elektronische Ressource


    Sprache :

    Englisch




    TACTICAL DECISION-MAKING IN AUTONOMOUS DRIVING BY REINFORCEMENT LEARNING WITH UNCERTAINTY ESTIMATION

    Hoel, Carl-Johan / Wolff, Krister / Laine, Leo | British Library Conference Proceedings | 2020


    Runtime Decision Making Under Uncertainty in Autonomous Vehicles

    Gautam, Vibhu / Gheraibia, Youcef / Alexander, Rob et al. | TIBKAT | 2021

    Freier Zugriff

    HIGHWAY TRAFFIC MODELING AND DECISION MAKING FOR AUTONOMOUS VEHICLE USING REINFORCEMENT LEARNING

    You, Changxi / Lu, Jianbo / Filev, Dimitar et al. | British Library Conference Proceedings | 2018