In autonomous driving, addressing occlusion scenarios is crucial yet challenging. Robust surrounding perception is essential for handling occlusions and aiding navigation. State-of-the-art models fuse LiDAR and Camera data to produce impressive perception results, but detecting occluded objects remains challenging. In this paper, we emphasize the crucial role of temporal cues in reinforcing resilience against occlusions in the bird’s eye view (BEV) semantic grid segmentation task. We proposed a novel architecture that enables the processing of temporal multi-step inputs, where the input at each time step comprises the spatial information encoded from fusing LiDAR and camera sensor readings. We experimented on the real-world nuScenes dataset and our results outperformed other baselines, with particularly large differences when evaluating on occluded and partially-occluded vehicles. Additionally, we applied the proposed model to downstream tasks, such as multi-step BEV prediction and trajectory forecasting of the ego-vehicle. The qualitative results obtained from these tasks underscore the adaptability and effectiveness of our proposed approach.


    Zugriff

    Zugriff prüfen

    Verfügbarkeit in meiner Bibliothek prüfen

    Bestellung bei Subito €


    Exportieren, teilen und zitieren



    Titel :

    TLCFuse: Temporal Multi-Modality Fusion Towards Occlusion-Aware Semantic Segmentation


    Beteiligte:


    Erscheinungsdatum :

    02.06.2024


    Format / Umfang :

    1783795 byte





    Medientyp :

    Aufsatz (Konferenz)


    Format :

    Elektronische Ressource


    Sprache :

    Englisch



    Hyperbolic Uncertainty Aware Semantic Segmentation

    Chen, Bike / Peng, Wei / Cao, Xiaofeng et al. | IEEE | 2024


    OCCLUSION AWARE SENSOR FUSION FOR EARLY CROSSING PEDESTRIAN DETECTION

    Palffy, Andras / Kooij, Julian F. P. / Gavrila, Dariu M. | British Library Conference Proceedings | 2019


    Occlusion aware sensor fusion for early crossing pedestrian detection

    Palffy, Andras / Kooij, Julian F. P. / Gavrila, Dariu M. | IEEE | 2019


    Machine Learning Multi-Modality Fusion Approaches Outperform Single-Modality & Traditional Approaches

    Garagic, Denis / Pelgrift, Daniel / Peskoe, Jacob et al. | IEEE | 2021


    Importance-Aware Semantic Segmentation for Autonomous Vehicles

    Chen, Bike / Gong, Chen / Yang, Jian | IEEE | 2019