Marine target searching is a complex task due to large search areas, unique signal propagation characteristics, and limited visibility, posing significant challenges for single-agent or homogeneous multi-agent systems. In response, we propose a novel hierarchical heterogeneous multi-agent (HHMA) framework designed for underwater search scenarios. This framework integrates three types of vehicles moving in different domains—unmanned aerial, surface, and underwater vehicles, effectively overcoming the limitations of single or double-agent configurations. We begin by elucidating the advantages of the HHMA system in target searching, providing the kinematic modeling, while also transforming sonar detecting data and defining the search problem. The mission is decomposed to three human-comprehensible subtasks that are adaptive to both environmental conditions and equipment capabilities: moving, target estimating and trajectory planning. The target estimating subtask is effectively modeled as a Markov Decision Process, retaining its memory capability. Additionally, we extend multi-agent reinforcement learning to multi-policy reinforcement learning, facilitating the training of interdependent policies. The efficacy of our approach is demonstrated through simulations, comparing it with rule-based methods. Simulation results underscore the significance of the HHMA system and validate the proposed training methodology.


    Access

    Check access

    Check availability in my library

    Order at Subito €


    Export, share and cite



    Title :

    Hierarchical Heterogeneous Multi-Agent Cross-Domain Search Method Based on Deep Reinforcement Learning


    Contributors:
    Dong, Shangqun (author) / Liu, Meiqin (author) / Dong, Shanling (author) / Zheng, Ronghao (author) / Wei, Ping (author)

    Published in:

    Publication date :

    2024-11-01


    Size :

    13130302 byte




    Type of media :

    Article (Journal)


    Type of material :

    Electronic Resource


    Language :

    English




    Multi-agent Cooperative Search based on Reinforcement Learning

    Sun, Yinjiang / Zhang, Rui / Liang, Wenbao et al. | IEEE | 2020


    Multi-Agent Mix Hierarchical Deep Reinforcement Learning for Large-Scale Fleet Management

    Huang, Xiaohui / Ling, Jiahao / Yang, Xiaofei et al. | IEEE | 2023


    Heterogeneous Multi-Agent Deep Reinforcement Learning for Cluster-Based Spectrum Sharing in UAV Swarms

    Xiaomin Liao / Yulai Wang / Yang Han et al. | DOAJ | 2025

    Free access