Introducing cooperative coded caching into small cell networks is a promising approach to reducing traffic loads. By encoding content via maximum distance separable (MDS) codes, coded fragments can be collectively cached at small-cell base stations (SBSs) to enhance caching efficiency. However, content popularity is usually time-varying and unknown in practice. As a result, cached content is anticipated to be intelligently updated by taking into account limited caching storage and interactive impacts among SBSs. In response to these challenges, we propose a multi-agent deep reinforcement learning (DRL) framework to intelligently update cached content in dynamic environments. With the goal of minimizing long-term expected fronthaul traffic loads, we first model dynamic coded caching as a cooperative multi-agent Markov decision process. Owing to the use of MDS coding, the resulting decision-making falls into a class of constrained reinforcement learning problems with continuous decision variables. To deal with this difficulty, we custom-build a novel DRL algorithm by embedding homotopy optimization into a deep deterministic policy gradient formalism. Next, to empower the caching framework with an effective trade-off between complexity and performance, we propose centralized, and partially and fully decentralized caching controls by applying the derived DRL approach. Simulation results demonstrate the superior performance of the proposed multi-agent framework.


    Zugriff

    Download


    Exportieren, teilen und zitieren



    Titel :

    Multi-Agent Reinforcement Learning for Cooperative Coded Caching via Homotopy Optimization


    Beteiligte:
    Xiongwei Wu (Autor:in) / Jun Li (Autor:in) / Ming Xiao (Autor:in) / P. C. Ching (Autor:in) / H. V. Poor (Autor:in)

    Erscheinungsdatum :

    01.08.2021


    Anmerkungen:

    oai:zenodo.org:5651716



    Medientyp :

    Aufsatz (Zeitschrift)


    Format :

    Elektronische Ressource


    Sprache :

    Englisch


    Klassifikation :

    DDC:    629 / 006




    Cooperative edge caching via multi agent reinforcement learning in fog radio access networks

    Chang, Q. (Qi) / Jiang, Y. (Yanxiang) / Zheng, F.-C. (Fu-Chun) et al. | BASE | 2022

    Freier Zugriff

    Multi-agent Cooperative Search based on Reinforcement Learning

    Sun, Yinjiang / Zhang, Rui / Liang, Wenbao et al. | IEEE | 2020


    Cooperative Edge Caching via Federated Deep Reinforcement Learning in Fog-RANs

    Zhang, M. (Min) / Jiang, Y. (Yanxiang) / Zheng, F.-C. (Fu-Chun) et al. | BASE | 2021

    Freier Zugriff