Heteroscedastic kernel ridge regression

In this paper we extend a form of kernel ridge regression (KRR) for data characterised by a heteroscedastic (i.e. input dependent variance) Gaussian noise process, introduced in Foxall et al. (in: Proceedings of the European Symposium on Artificial Neural Networks (ESANN-2002), Bruges, Belgium, April 2002, pp. 19–24). It is shown that the proposed heteroscedastic kernel ridge regression model can give a more accurate estimate of the conditional mean of the target distribution than conventional KRR and also provides an indication of the spread of the target distribution (i.e. predictive error bars). The leave-one-out cross-validation estimate of the conditional mean is used in fitting the model of the conditional variance in order to overcome the inherent bias in maximum likelihood estimates of the variance. The benefits of the proposed model are demonstrated on synthetic and real-world benchmark data sets and for the task of predicting episodes of poor air quality in an urban environment.