The cooperation in mixed cooperative-competitive tasks has drawn significant attention in multi-agent deep reinforcement learning. Agents need to cooperate with their teammates while competing against their opponents. However, most existing works treat the cooperative agents and competitive agents equally as they perform the same operation on all the agents. As a result, without distinguishing between cooperative and competitive agents, they may suffer from information disorder in learning an optimally cooperative policy and struggle to decide on the next step action. To address the above issues, we decompose the final Q-value into a weighted combination of three parts: the Q-values of the cooperative agents, the competitive group, and the current agent. A theoretical proof of the correctness of the decomposition is provided. With this decomposition, we are able to consider cooperative and competitive agents separately. Accordingly, we propose a multi-agent actor-critic algorithm called actor-double-attention-critic (ADAC) under centralized training and decentralized execution according to the decomposition. In ADAC, networks with group-specific attention and an attentional weighting network are specially designed. With the designed double-attention structure, ADAC can capture the distributions from different agents and improve cooperation performance. Extensive experiments are conducted in three scenarios with nine settings against six representative methods. The results demonstrate the superiority of the proposed ADAC model against state-of-the-art methods in various mixed cooperative-competitive tasks. The code is available at https://github.com/CrazyBayes/ADAC
ADAC: Actor-Double-Attention-Critic for Multi-Agent Cooperation in Mixed Cooperative-Competitive Environments
IEEE Transactions on Intelligent Transportation Systems ; 26 , 7 ; 9579-9592
2025-07-01
3861215 byte
Article (Journal)
Electronic Resource
English
ArXiv | 2021
|Factored Multi-Agent Soft Actor-Critic for Cooperative Multi-Target Tracking of UAV Swarms
DOAJ | 2023
|Multi-agent deep reinforcement learning with actor-attention-critic for traffic light control
SAGE Publications | 2024
|Actor-Critic Policy Learning in Cooperative Planning
AIAA | 2010
|Actor-Critic Policy Learning in Cooperative Planning
British Library Conference Proceedings | 2010
|