Physically Plausible Spectral Reconstruction from RGB Images

Research output: Contribution to conferencePaper

1 Citation (Scopus)
8 Downloads (Pure)

Abstract

Recently Convolutional Neural Networks (CNN) have been used to reconstruct hyperspectral information from RGB images, and this spectral reconstruction problem (SR) can often be solved with good (low) error. However, little attention has been paid on whether these models' behavior can adhere to physics. We show that the leading CNN method introduces unexpected 'colorimetric errors', which means the recovered spectra do not reproduce ground-truth RGBs, and sometimes this discrepancy can be large. The problem is further compounded by exposure change. Indeed, most CNN models over-fit to fixed exposure and we demonstrate that this can result in poor performance when exposure varies.In this paper we show how CNN learning can be extended so that the physical plausibility of SR is enforced. Remarkably, our physically plausible CNN solutions advance both spectral and colorimetric performance of the original network, while the application of data augmentation trades off the network performance for model stability against varying exposure.

Original languageEnglish
Pages2257-2266
Number of pages10
DOIs
Publication statusPublished - 28 Jul 2020
Event2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) - Seattle, WA, USA
Duration: 14 Jun 202019 Jun 2020

Conference

Conference2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)
Period14/06/2019/06/20

Cite this