Adaptive Traffic Signal Control methods based on Reinforcement Learning have been applied successfully to reach state-of-the-art results in simulation. However, most recent works in the area use model-free methods, which learn value and/or policy functions directly through environment in-teractions without a dynamics model. Model-free methods have several potential drawbacks: (1) difficulties in generalization to dynamics not seen in training, as well as difficulties in transfer to different intersection geometries; (2) low sample-efficiency, resulting in high resource requirements for training; (3) often, reliance on difficult-to-interpret function approximators such as neural networks. In this work, we demonstrate experimentally that Monte Carlo Tree Search (MCTS) with a simple queue and platoon-propagation dynamics model can reach lower delay on a single intersection than the standard model-free Deep Q-Networks (DQN) algorithm. Moreover, we provide empirical results that illustrate that model-based receding-horizon planning can alleviate many of the drawbacks of the model-free methods listed above: better generalization characteristics; more direct and less resource-demanding transfer to different intersection geometries and phasing schemes; and simpler visualization and interpretation, aiding modification and debugging.


    Access

    Check access

    Check availability in my library

    Order at Subito €


    Export, share and cite



    Title :

    A Case for Monte Carlo Tree Search in Adaptive Traffic Signal Control: Modifiability, Interpretability and Generalization


    Contributors:


    Publication date :

    2023-09-24


    Size :

    919827 byte





    Type of media :

    Conference paper


    Type of material :

    Electronic Resource


    Language :

    English



    Monte Carlo Tree Search-Based Mixed Traffic Flow Control Algorithm for Arterial Intersections

    Cheng, Yanqiu / Hu, Xianbiao / Tang, Qing et al. | Transportation Research Record | 2020


    Adaptive Reward for CAV Action Planning using Monte Carlo Tree Search

    Patel, Dhruvkumar / Zalila-Wenkstern, Rym | IEEE | 2021


    Multi-Objective Monte-Carlo Tree Search based aerial maneuvering control

    Xiang Chen / Xinguo Zhang / Weijia Wang et al. | IEEE | 2016


    MONTE CARLO TREE SEARCH METHODS FOR TELESCOPE TASKING

    Fedeler, Samuel / Holzinger, Marcus | TIBKAT | 2020


    Monte Carlo Tree Search Methods for Telescope Tasking

    Fedeler, Samuel / Holzinger, Marcus | AIAA | 2020