RGB-Thermal semantic segmentation is crucial for applications like robotic inception, autonomous driving, and video surveillance. However, small and imbalanced datasets in this field create training bias. To address this, we propose a new data augmentation scheme. We mix up the cross-modal information to smooth the decision boundaries from class to class, modal to modal using CutMix and copy-paste. Using the two method, RGB data and Thermal data are exchanged. Applying the RandAugment method to augment RGB and Thermal images, we select 2 transformations from a pool of 15, including a new transformation tailored for RGB-Thermal datasets. This innovative transformation mimics modal failure by fading a randomly selected region in either RGB or thermal images. Our experiments confirm the effectiveness of the proposed augmentation scheme, leading to a observable performance boost. Specifically, we observe improvements of 4.7% in mAcc and 6.1% in mIoU compared to the baseline.
Cross-modal Augmentation: A Data Augmentation Scheme for RGB-Thermal Semantic Segmentation
2023-10-11
4475282 byte
Conference paper
Electronic Resource
English
Leveraging Smooth Deformation Augmentation for LiDAR Point Cloud Semantic Segmentation
IEEE | 2024
|