Decision-making for urban autonomous driving is challenging due to the stochastic nature of interactive traffic participants and the complexity of road structures. Although reinforcement learning (RL)-based decision-making schemes are promising to handle urban driving scenarios, they suffer from low sample efficiency and poor adaptability. In this paper, we propose the Scene-Rep Transformer to enhance RL decision-making capabilities through improved scene representation encoding and sequential predictive latent distillation. Specifically, a multi-stage Transformer (MST) encoder is constructed to model not only the interaction awareness between the ego vehicle and its neighbors but also intention awareness between the agents and their candidate routes. A sequential latent Transformer (SLT) with self-supervised learning objectives is employed to distill future predictive information into the latent scene representation, in order to reduce the exploration space and speed up training. The final decision-making module based on soft actor-critic (SAC) takes as input the refined latent scene representation from the Scene-Rep Transformer and generates decisions. The framework is validated in five challenging simulated urban scenarios with dense traffic, and its performance is manifested quantitatively by substantial improvements in data efficiency and performance in terms of success rate, safety, and efficiency. Qualitative results reveal that our framework is able to extract the intentions of neighbor agents, enabling better decision-making and more diversified driving behaviors.


    Access

    Check access

    Check availability in my library

    Order at Subito €


    Export, share and cite



    Title :

    Augmenting Reinforcement Learning With Transformer-Based Scene Representation Learning for Decision-Making of Autonomous Driving


    Contributors:
    Liu, Haochen (author) / Huang, Zhiyu (author) / Mo, Xiaoyu (author) / Lv, Chen (author)

    Published in:

    Publication date :

    2024-03-01


    Size :

    6814361 byte




    Type of media :

    Article (Journal)


    Type of material :

    Electronic Resource


    Language :

    English




    A Decision-Making Method for Connected Autonomous Driving Based on Reinforcement Learning

    Zhang, Mingheng / Wan, Xing / Lv, Xinfei et al. | British Library Conference Proceedings | 2020


    A Decision-Making Method for Connected Autonomous Driving Based on Reinforcement Learning

    Zhang, Mingheng / Lv, Xinfei / Wan, Xing et al. | SAE Technical Papers | 2020


    Human-Like Decision Making and Planning for Autonomous Driving with Reinforcement Learning

    Zong, Ziqi / Shi, Jiamin / Wang, Runsheng et al. | IEEE | 2023


    A Decision-Making Method for Connected Autonomous Driving Based on Reinforcement Learning

    Zhang, Mingheng / Wan, Xing / Lv, Xinfei et al. | British Library Conference Proceedings | 2020