The utility of multiple reinforcement learning (RL) agents collaboratively training within a shared environment, all working towards common objectives, is increasingly evident within the Internet of Vehicles (IoV). The multi-agent Advantage Actor-Critic (MA2C) algorithm is a prominent example of such a Multi-Agent Reinforcement Learning (MARL) system. However, MA2C requires agents to share policies, such as pairs of states and actions and even trained models, among neighboring agents, to overcome the challenge of agents having only partial observations. Unfortunately, this requirement amplifies the communication overhead and raises privacy concerns. Federated learning (FL), as a privacy-preserving machine learning method, can be applied in the MARL context with a central server aggregating the weights of the agents' models. However, this technique assumes that all agents are capable of executing identical actions, which may be impractical. In this paper, we introduce a novel FL A2C algorithm called Advantage Actor Federated Critic (A2FC). The proposed algorithm streamlines the aggregation of agents' critic models while offloading the training of actor models to the individual agents' local machines. An experiment conducted in an adaptive traffic signal control (ATSC) system demonstrates the method's effectiveness in personalizing agents' actions, preserving agents' privacy during training, and mitigating communication overhead issues.


    Access

    Check access

    Check availability in my library

    Order at Subito €


    Export, share and cite



    Title :

    Federated Multi-Agent Reinforcement Learning for Heterogeneous Action Spaces


    Contributors:


    Publication date :

    2024-06-24


    Size :

    884736 byte





    Type of media :

    Conference paper


    Type of material :

    Electronic Resource


    Language :

    English



    Communication-efficient and federated multi-agent reinforcement learning

    Krouka, M. (Mounssif) / Elgabli, A. (Anis) / Issaid, C. B. (Chaouki Ben) et al. | BASE | 2022

    Free access



    Vehicular cooperative perception through action branching and federated reinforcement learning

    Abdel-Aziz, M. K. (Mohamed K.) / Perfecto, C. (Cristina) / Samarakoon, S. (Sumudu) et al. | BASE | 2022

    Free access

    Vehicular Cooperative Perception Through Action Branching and Federated Reinforcement Learning

    Abdel-Aziz, Mohamed K. / Perfecto, Cristina / Samarakoon, Sumudu et al. | ArXiv | 2020

    Free access