Traditional millimeter-wave beam selection or prediction algorithms typically rely on beam scanning measurements at the transceivers, incurring substantial training overhead and exhibiting limited adaptability in diverse environments. Recent efforts have aimed to mitigate these challenges by incorporating sensing information, thereby reducing or eliminating the need for extensive beam training. However, existing works predominantly concentrate on exploiting a single sensing modality and often overlook the potential benefits of utilizing historical sensing information. In this paper, we introduce an intelligent beam prediction framework that leverages a deep integration of multimodal sensing data, encompassing GPS, camera, radar, and LiDAR data. The design proposed involves the application of customized deep neural networks to extract features from camera, radar, and LiDAR data. These extracted features, combined with user position and selected beam index, are concatenated to form an aggregated feature vector at each time instance. Subsequently, a time series of these concatenated feature vectors is utilized to exploit temporal correlation for beam prediction through a dedicated long short-term memory network module. Numerical simulations confirm the effectiveness of the proposed design and its superiority over several considered state-of-the-art baselines.


    Access

    Check access

    Check availability in my library

    Order at Subito €


    Export, share and cite



    Title :

    Multimodal Deep Learning Empowered Millimeter-Wave Beam Prediction


    Contributors:
    Shi, Binpu (author) / Li, Min (author) / Zhao, Ming-Min (author) / Lei, Ming (author) / Li, Liyan (author)


    Publication date :

    2024-06-24


    Size :

    1239005 byte





    Type of media :

    Conference paper


    Type of material :

    Electronic Resource


    Language :

    English



    Digital Twin-Empowered Routing Management for Reliable Multi-Hop Millimeter Wave V2X

    Roongpraiwan, Supat / Li, Zongdian / Yu, Tao et al. | IEEE | 2024


    Deep Learning for Fast Beam Tracking using RSRP in Millimeter Wave MIMO Systems

    Zhang, Jiankun / Wang, Hao / Du, Guanglong et al. | IEEE | 2022


    Deep Learning Empowered Secure RIS-Assisted Non-Terrestrial Relay Networks

    Huang, Chong / Chen, Gaojie / Zhou, Yitong et al. | IEEE | 2022


    Millimeter Wave Antenna with Scanning Beam

    Derkach, V. N. / Vertiy, A. A. | British Library Online Contents | 1999


    Millimeter Wave Base Stations with Cameras: Vision-Aided Beam and Blockage Prediction

    Alrabeiah, Muhammad / Hredzak, Andrew / Alkhateeb, Ahmed | IEEE | 2020