Reinforcement learning (RL) is a promising data-driven approach for adaptive traffic signal control (ATSC) in complex urban traffic networks, and deep neural networks further enhance its learning power. However, the centralized RL is infeasible for large-scale ATSC due to the extremely high dimension of the joint action space. The multi-agent RL (MARL) overcomes the scalability issue by distributing the global control to each local RL agent, but it introduces new challenges: now, the environment becomes partially observable from the viewpoint of each local agent due to limited communication among agents. Most existing studies in MARL focus on designing efficient communication and coordination among traditional Q-learning agents. This paper presents, for the first time, a fully scalable and decentralized MARL algorithm for the state-of-the-art deep RL agent, advantage actor critic (A2C), within the context of ATSC. In particular, two methods are proposed to stabilize the learning procedure, by improving the observability and reducing the learning difficulty of each local agent. The proposed multi-agent A2C is compared against independent A2C and independent Q-learning algorithms, in both a large synthetic traffic grid and a large real-world traffic network of Monaco city, under simulated peak-hour traffic dynamics. The results demonstrate its optimality, robustness, and sample efficiency over the other state-of-the-art decentralized MARL algorithms.


    Access

    Check access

    Check availability in my library

    Order at Subito €


    Export, share and cite



    Title :

    Multi-Agent Deep Reinforcement Learning for Large-Scale Traffic Signal Control


    Contributors:
    Chu, Tianshu (author) / Wang, Jie (author) / Codeca, Lara (author) / Li, Zhaojian (author)


    Publication date :

    2020-03-01


    Size :

    3444177 byte




    Type of media :

    Article (Journal)


    Type of material :

    Electronic Resource


    Language :

    English



    Traffic signal control method based on multi-agent deep reinforcement learning

    LIN FENG / HE SHUAI / SHAO LANG et al. | European Patent Office | 2025

    Free access

    Multi-Agent Deep Reinforcement Learning for Decentralized Cooperative Traffic Signal Control

    Zhao, Yang / Hu, Jian-Ming / Gao, Ming-Yang et al. | ASCE | 2020




    Large-Scale Mixed-Traffic and Intersection Control using Multi-agent Reinforcement Learning

    Liu, Songyang / Fan, Muyang / Li, Weizi et al. | ArXiv | 2025

    Free access