There exists spectral gap between visible (VIS) and near infrared (NIR) images causing dissimilar intensity according to reflection property of objects and materials. Therefore, it has a limit of applying traditional stereo matching to cross spectral disparity estimation. In this paper, we propose cross spectral disparity estimation from VIS and NIR paired images using disentangled representation and reversible neural networks. We build a supervised learning framework based on reversible blocks to extract scene features robust against the spectral gap. Reversible blocks decompose features into scene and style components to bridge the spectral gap between VIS and NIR images. We perform stereo matching on the scene component to get an initial disparity map by a 3D convolutional neural network. To generate clear edges in the disparity map, we use a semantic segmentation network as auxiliary information to refine the initial disparity map. Besides, to consider the lack of the ground truth, we synthesize reference disparity maps with guided image filtering. Experimental results demonstrate that the proposed method achieves accurate edges in disparity along object boundaries and outperforms the state-of-the-art methods in both visual comparison and quantitative measurements.
Cross Spectral Disparity Estimation From VIS and NIR Paired Images Using Disentangled Representation and Reversible Neural Networks
IEEE Transactions on Intelligent Transportation Systems ; 24 , 5 ; 5326-5336
01.05.2023
1883271 byte
Aufsatz (Zeitschrift)
Elektronische Ressource
Englisch
Motion Estimation from Disparity Images
British Library Conference Proceedings | 2001
|Motion estimation from disparity images
IEEE | 2001
|Enhanced disparity estimation in stereo images
British Library Online Contents | 2015
|