On-line learning methods have been applied successfully in multi-agent systems to achieve coordination among agents. Learning in multi-agent systems implies in a non-stationary scenario perceived by the agents, since the behavior of other agents may change as they simultaneously learn how to improve their actions. Non-stationary scenarios can be modeled as Markov Games, which can be solved using the Minimax-Q algorithm a combination of Q-learning (a Reinforcement Learning (RL) algorithm which directly learns an optimal control policy) and the Minimax algorithm. However, finding optimal control policies using any RL algorithm (Q-learning and Minimax-Q included) can be very time consuming. Trying to improve the learning time of Q-learning, we considered the QS-algorithm. in which a single experience can update more than a single action value by using a spreading function. In this paper, we contribute a Minimax-QS algorithm which combines the Minimax-Q algorithm and the QS-algorithm. We conduct a series of empirical evaluation of the algorithm in a simplified simulator of the soccer domain. We show that even using a very simple domain-dependent spreading function, the performance of the learning algorithm can be improved.


    Access

    Download


    Export, share and cite



    Title :

    Experience generalization for multi-agent reinforcement learning



    Publication date :

    2001-01-01



    Type of media :

    Conference paper


    Type of material :

    Electronic Resource


    Language :

    English


    Classification :

    DDC:    629 / 006



    Traffic signal priority control based on shared experience multi‐agent deep reinforcement learning

    Zhiwen Wang / Kangkang Yang / Long Li et al. | DOAJ | 2023

    Free access

    Traffic signal priority control based on shared experience multi‐agent deep reinforcement learning

    Wang, Zhiwen / Yang, Kangkang / Li, Long et al. | Wiley | 2023

    Free access

    Brainstorming Multi-Agent Reinforcement Learning for Multi-Vehicles Games

    Liu, Yingxiang / Li, Hao / Zhu, Xuefeng | IEEE | 2021


    Multi Agent Reinforcement Learning for Gridworld Soccer Leadingpass

    Sari, S.C. / Prihatmanto, A.S. / Kuspriyanto | British Library Conference Proceedings | 2013


    ATFM computational agent based on reinforcement learning aggregating human expert experience

    Crespo, A. M. F. / Li Weigang, / Timoszczuk, A. P. | IEEE | 2011