Physically Plausible Spectral Reconstruction from RGB Images

Research output: Contribution to conferencePaperpeer-review

12 Citations (Scopus)
15 Downloads (Pure)


Recently Convolutional Neural Networks (CNN) have been used to reconstruct hyperspectral information from RGB images, and this spectral reconstruction problem (SR) can often be solved with good (low) error. However, little attention has been paid on whether these models' behavior can adhere to physics. We show that the leading CNN method introduces unexpected 'colorimetric errors', which means the recovered spectra do not reproduce ground-truth RGBs, and sometimes this discrepancy can be large. The problem is further compounded by exposure change. Indeed, most CNN models over-fit to fixed exposure and we demonstrate that this can result in poor performance when exposure varies.In this paper we show how CNN learning can be extended so that the physical plausibility of SR is enforced. Remarkably, our physically plausible CNN solutions advance both spectral and colorimetric performance of the original network, while the application of data augmentation trades off the network performance for model stability against varying exposure.

Original languageEnglish
Number of pages10
Publication statusPublished - 28 Jul 2020
Event2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops - Seattle, United States
Duration: 14 Jun 202019 Jun 2020


Conference2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops
Abbreviated titleCVPRW
Country/TerritoryUnited States

Cite this