RGB-Thermal semantic segmentation is crucial for applications like robotic inception, autonomous driving, and video surveillance. However, small and imbalanced datasets in this field create training bias. To address this, we propose a new data augmentation scheme. We mix up the cross-modal information to smooth the decision boundaries from class to class, modal to modal using CutMix and copy-paste. Using the two method, RGB data and Thermal data are exchanged. Applying the RandAugment method to augment RGB and Thermal images, we select 2 transformations from a pool of 15, including a new transformation tailored for RGB-Thermal datasets. This innovative transformation mimics modal failure by fading a randomly selected region in either RGB or thermal images. Our experiments confirm the effectiveness of the proposed augmentation scheme, leading to a observable performance boost. Specifically, we observe improvements of 4.7% in mAcc and 6.1% in mIoU compared to the baseline.
Cross-modal Augmentation: A Data Augmentation Scheme for RGB-Thermal Semantic Segmentation
11.10.2023
4475282 byte
Aufsatz (Konferenz)
Elektronische Ressource
Englisch
Leveraging Smooth Deformation Augmentation for LiDAR Point Cloud Semantic Segmentation
IEEE | 2024
|