Stable and reliable wireless communications are absolutely essential in achieving safe, efficient, and comfortable autonomous driving systems, and require more channel resources. Cognitive radio (CR) alleviates the spectrum shortage by exploiting spectrum resources that are not used by primary users (PUs). Conventional CR-based vehicle-to-everything (V2X) communications still suffer from spectrum scarcity when the demands for wireless resources are excessive. Most previous analyses of this problem have adopted too simple models for V2X that assume constant vehicle velocity and that coarsely discretize vehicle positions. More realistic models for V2X are too topologically complex and too dynamically variant to find stable solutions of optimization problems by using conventional off-policy-based reinforcement learning (RL) algorithms such as Q-learning and deep Q-network (DQN) based on reply memories. Considering the above situation, this paper firstly builds a precise and realistic autonomous driving testbed by 3D engine. In the testbed, all vehicles move in an autopilot mode with random velocity and direction, which can provide more realistic traffic flow, positions, and motion for all vehicles. Due to the constrain of Kullback-Leibler divergence between updated and previous policies, on-policy RL algorithms such as proximal policy optimization (PPO) can find stable solutions to complex optimization problems. Inspired by this property, we expand the single-agent RL algorithm PPO into a multi-agent algorithm MA-PPO (multi-action proximal policy optimization). Computer simulations using the new testbed show the MA-PPO algorithm can gradually converge to more stable solutions and can achieve more steady and efficient data transmission than DQN.


    Zugriff

    Zugriff prüfen

    Verfügbarkeit in meiner Bibliothek prüfen

    Bestellung bei Subito €


    Exportieren, teilen und zitieren



    Titel :

    Reinforcement Learning-Based Cognitive Radio Transmission Scheduling in Vehicular Systems


    Beteiligte:
    Li, Yun (Autor:in) / Chang, Yuyuan (Autor:in) / Fukawa, Kazuhiko (Autor:in) / Kodama, Naoki (Autor:in)


    Erscheinungsdatum :

    01.06.2023


    Format / Umfang :

    2626266 byte





    Medientyp :

    Aufsatz (Konferenz)


    Format :

    Elektronische Ressource


    Sprache :

    Englisch



    Deep Reinforcement Learning based Adaptive Transmission Control in Vehicular Networks

    Liu, Mingyuan / Quan, Wei / Yu, Chengxiao et al. | IEEE | 2021


    AoI-Aware Dynamic User Scheduling in Vehicular Networks Based on Soft Reinforcement Learning

    Huang, Zhisen / Guo, Chongtao / Chen, Jiayi et al. | IEEE | 2023


    Robust beamforming for cognitive radio based vehicular communication

    Alam, Md Monzurul / Bhattarai, Sudeep / Hong, Liang et al. | IEEE | 2013


    Multi-Objective Reinforcement Learning for Cognitive Radio-Based Satellite Communications

    Ferreira, Paulo Victor R. / Paffenroth, Randy / Wyglinski, Alexander M. et al. | NTRS | 2016


    VEHICULAR RADIO WAVE TRANSMISSION COMPONENT

    KITAYAMA KEIICHI / MORI SATOSHI / NARITA YASUHIDE et al. | Europäisches Patentamt | 2019

    Freier Zugriff